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ABSTRACT 

This report describes results of a study of academic 
performance by American Indian elementary school students with 
limited-English proficiency CLEP). The study was designed to 
complement the National Longitudinal Evaluation of the Effectiveness 
of Services for Language Minority Limited-English-Proficient 
Students, which was also conducted for the Department of Education, 
but which did not include Native American students. This report 
summarizes the results of the second year of the two-year study of 
performance of LEP American Indian students. It includes results of 
two years of on-site data collection in a sample of eight projects 
visited. Schools visited include public schools, tribally controlled 
schools, and one school controlled by the federal Bureau of Indian 
Affairs. Researchers assessed the extent to which Indian languages 
and English are used at home and during school. Overall, the students 
participating in the study scored substantially below the national 
average on standardized achievement tests in mathematics and English. 
Indian students scored lower on math subtests than did students in 
the main LEP study, even though their English oral proficiency 
ratings were higher. Several variables are examined but the report 
suggests Indian students 9 low test scores are largely due to lack of 
exposure to English and lack of home support for educational 
achievement. A final section examines the implications and areas for 
further study. Graphs, tables and references are included. There is a 
glossary and appendixes describing the study's design, methodology, 
data, and the "technical advisory/ panel members." CAuthor/TES) 
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"PREFACE 

This report describes the academic performance of elementary grade level 
limited-English-proficient Indian students attending school on or near 
Indian reservations* The study was designed as a complement to the 
"National Longitudinal Evaluation of the Effectiveness of Services for 
Language-Minority Limited-English-Proficient Students'* ,hich was also 
conducted for the Department of Education but did not contain a sample of 
Native American students. The report summarizes the results of the second 
year of a two-year study. (The results of the first year are summarized in 
Rudes et al. , 1988.) Included are the results of two years of on-site data 
collection in a sample of 8 of the 23 projects visited in the first year -of 
the study. The data collection and analyses were performed by Development 
Associates , Inc., In affiliation with the Research Triangle Institute, 
during the years 1987-1988. 
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Chapter 1. INTRODUCTION 1 

This report describes the findings from the second year of data collection 

and analysis for the "National Evaluation of Instructional Services for 

2 

Limited-English-Proficient Native American Students. H It focuses on the 
academic performance of limited-English-prof icient (LEP) American Indian 
students in the elementary grades in isolated rural schools located on or near 
Indian reservations. To obtain a sizeable population of American Indian LEP 
students, the study focused on schools participating in Title VII (bilingual 
education) projects for Native American students. 

Overall, the students participating in the study scored substantially below 
the national average on standardized achievement tests of mathematics and 
English language arts. In this report, test results are examined in detail 
together with data on the instructional services provided to these students and 
the characteristics of the community, home, and school context in which 
instruction is provided. 

Before proceeding to the presentation of study findings, a brief overview of 
the history of Indian education and instructional services for limited- 
English-proficient students in the United States will provide the context for 
study result 8. 



^Definitions of special terms and symbols used in this chapter are provided in 
Anpendix A. 

^The primary objective of the first part of the study was to describe the 
instructional services p^ovldcv i N limited-English-proficient Native American 
students ia the elementary grades. The findings from that portion of the study 
were reported iif in Rudes e^i aJL (1988). 
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A. HISTORY AND BACKGROUND 



The relationship of the federal government and the American Indian has 
changed many times in the past 200 years. Since the early development of the 
United States, it has been official policy for the federal government to relate 
to Indian people on a nation-to-nation basis. Although treated as sovereign, 
Indian nations were subject to legal control through treaties with the federal 
government. By 1870, the period of treaty making had ended; encroachment onto 
Indian land escalated, and federal domination grew. There began a resettlement 
period as entire tribes were moved from their ancestral lands to regions 
considered unsuited for other use. 

Education became a major focus of federal Indian policy, as administered 
through the Bureau of Indian Affairs (BIA), in 1870 when Congress first 
appropriated funds on a regular basis to provide educational services for 
certain Indian students in the United States. Prior to this, educational 
funding had been limited to treaty obligations and to a Civilization Fund. 
Currently, the Snyder Act of 1921 (25 U.S.C. 13) provides the basic authority 
under which the BIA provides educational and other services to federally 
recognized Indians. These services include direct operation of schools, 
support of tribal ly-operated schools, and financial assistance to public 
schools serving Indian children. 

With passage of the Johnson 0 T Malley Act in 1934 and key amendments in 1936, 
the Secretary of Interior was empowered to expend federal funds through 
contracts with state or local agencies for the education of Indian children. 
The Indian Self-Determinatictk Act of 1975 (P.L. 93-638) abends the Johnson 
O'Malley Act to provide for maximum Indian participation in Indian education 
and provides for tuition payment for Indian children. As amended, the Johnson 
O'Malley Act authorizes funding for supplemental programs for eligible Indian 
students in public schools, age three years through grade twelve, to meet their 
special and unique educational needs as determined by contracting agencies and 
local adian education committees. 



A major thread running through the history of Indian affairs has been the 
government policy of "assimilation, " and education has been its most important 
single means. Federal responsibility in education developed during the treaty 
period when approximately 400 treaties were signed, many of Which contained 
provisions for education. Judicial decisions, Congressional legislation, and 
executive orders steadily increased the federal responsibility to educate the 
American Indian. The actual response, however, has varied. Over time, the 
government has directed subsidies to mission schools, established a national 
Indian school system, supported state public school system, supported 
alternative school systems, and enacted legislation intended to meet specific 
needs of Indian students. 

The middle of the twentieth century (1940s-1950s) saw a period of 
termination of federal trust responsibility to Indian tribes and the trust 
relationship with certain tribes was ended. This was followed by a period of 
relocating Indians to urban areas for training and jobs (1940s-1960s). Many 
left their reservations and villages during this time. 

Recently, the federal policy toward Indians has been one of self- 
determination. Made official during the Nixon administration, its origins go 
back to President Johnson's White House Task Force on Indian Affairs (appointed 
in 1966) and, in Congress, to the introduction by Senator McGovera (D-SD) of a 
concurrent resolution to increase Indian economic development and self- 
determination. In 1968, President Johnson sent a message to Congress 
recommending that Indians become involved in their own affairs, and, in 1970, 
Congress laid out a policy of Indian self-determination, the effects of which 
were felt through the 1970s. This policy was reaffirmed by President Reagan in 
the "White House Indian Policy Statement n of January 14, 1983. 

Thus, the late 1960s was an important period in the development of Indian 
education. The political and social atmosphere supported positive growth. The 
termination period of the 1940s and 1950s had made Indian people suspicious of 
the federal government, but they pushed for greater federal responsibility in 
Indian affairs. 
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Systematic documentation of conditions among Indians and Alaska Natives 
began with the "Meriam Report" (The Problem of Indian Administration) issued in 
1923; it is widely considered to be the first comprehensive review of Indian 
affairs. Concluding that the "first and foremost need in Indian, education is 
to atop removing Indian children from their families," the report suggested 
that local day schools be emphasized in place of boarding schools* Forty-one 
years later, in 1969, the second major study of Indian education was released. 
The tone and findings of the "Kennedy Report" are reflected in its title, 
Indian Education: A National Tragedy - A National Challenge . The report 
provided documented evidence, based upon extensive investigation, that both the 
BIA and the public school system were failing* In both educational systems, 
Indian students were characterized by high dropout rates, low academic 
achievement, and low self -concept; there was also a lack of community and 
parental involvement, and a dearth of Indian teachers and school 
administrators. Sixty recommendations for improving Indian education were 
made* However, the authors asserted that before any recommendations could be 
acted upon: 

The Federal Government must commit itself to a national policy of 
educational excellence for Indian children, maximum participation 
and control by Indian adults and communities, and the development 
of new legislation and substantial Increases in appropriations. •• 

In the years following the release of the "Kennedy Report," numerous 
changes were made in federal legislation and pulley with an aim toward 
improving Indian education overall* These included the enactment of laws 
such as the Indian Education Act (Title IV of P.L. 92-318) in 1974 and the 
Indian Self-Determination and Educational Assistance Act (P.L. 93-638) in 
1975, as well as the amendment of existing laws such as the Johnson O'Malley 
Act of 1934. However, probably more important in the long run than the new 
amendments and laws were the changes which were wrought in the Indian 
community by the new policies emanating from Washington. For the first time 
since the federal government assumed responsibility for educating Indian 
children in 1865-70, Indian parents found that they could exercise an element 
of control over the education which their children received. 
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Another federal law Which has had substantial impact on the education of 
Indian students, specifically those who are United in their English 
proficiency, was ESEA Title VII (the "Bilingual Education Act"). ESEA Title 
VII was enacted into law in 1968. Originally designed to help local school 
districts develop programs of language-related services to improve the 
education of limited-English-proficient students by funding demonstration 
projects. Title VII rapidly grew into a major funding source for the general 
operation of such programs. It also became a major tool for districts to use 
in complying with the 1974 decision by the Supreme Court in Lau v. Nichols 
(414 U..S. 563) which stated that school districts are required, under Title 
VI of the Civil Rights Act of 1964, to provide limited-English-proficient 
students with instructional services designed to overcome their English- 
language deficiency* In the early years of Title VII, few projects serving 
Indian student s were funded. However , since 1974 rr the year of the Lau 
decision and the year of the passage of the Indian Education Act — the 
number of projects funded by Title VII to serve Indian students has increased 
significantly. Furthermore, as part of the Educational Amendments of 1978, 
the definition of the target population for Title VII was revised 
specifically to include Native American students as beneficiaries under the 
act. 

IB B. OVERVIEW OF STUDY PURPOSE AND DESIGN 

The motivation for conducting this study grew, in large part, out of the 
work being carried out by Development Associates for the "National 
Longitudinal Evaluation of the Effectiveness of Services for Language- 
Minority Limited-English-Proficient Students" (hereafter referred to as the 
LM-LEP Study). The sample of 12,000 students participating in that study 
was selected to be nationally representative, and thus consisted of large 
numbers of Hispanic, Chinese, and Southeast Asian students, but only a small 
number of Native American students. Because of the special interest and 
/ responsibilities of the federal government vis-a-vis instructional services 

for Native American students, the U.S. Department of Education determined 
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that a separate study, replicating the instruments and procedures of the 
study described above, should be carried out with a sample of Native 
American students* 

The primary objective of the first part of this study was to describe 
the instructional services provided to limited-English-proficient Native 
American students in the elementary grades* The findings from that portion 
of the study were, reported upon in Rudes et al • (1988)* The primary 
objective of the second part of the study — the focus of this report — was 
to acquire a fuller understanding of the academic performance of Native 
American LEP students in the elementary grades attending rural schools on or 
near Indian reservations* 

- .The. basic research .plan .called _jfor_data to, be collected on Jtrwp cohorts 

of students in a national sample of schools served by Title VII projects 

which served Native American elementary school students. 1 The first 

cohort (Cohort A) consists of students who were in grade 1 during the 

1985-86 school year* The second cohort (Cohort B) consists of students who 

vere in grade 3 that year. During the first year of the study, the school 

sample was drawn based on a review of Title VII grant applications at the 

Office of Bilingual Education and Minority Languages Affairs (OBEMLA) and 

telephone and mail contacts with all 58 Title VII projects identified as 

serving primarily Native American students* Twenty-three projects were 

2 

selected for on-site data collection* These 23 projects included 32 



1 The school sample for this study was selected from among those schools which 
had ongoing Title VH projects serving primarily Native American students* This 
was done in order to facilitate the identification of schools with large numbers 
of limited-English-proficient Native American students* However, while all of the 
schools in the study had Title Vll-funded projects, not all of the students in the 
study's student sample received Title VII services* Rather, all Native American 
students in grades 1 and 3 in these schools were included, regardless of the 
instructional services they were receiving* 

^The contacts with all projects were for the purpose of identifying the sample of 
projects to be visited* Because of the high cost of data collection in Alaska, 
the decision was made by the U*S* Department of Education to exclude projects in 
Alaska fro* the on-site data collection for this study* 
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schools and served a total of 1588 first and third grade Native American 
students Who cave fro* 16 different tribal groups, and from 18 different 
native language backgrounds* Although not a probability sample, the 
students In that study constitute a reasonable sample of Indian students 
attending rural schools on or near Indian reservations* 

For the second year of the study, 8 of the 23 projects visited during 
the first year were selected for on-site data collection* These 8 projects 
included 11 schools and served a total of 498 second and fourth grade Native 
American students who came from 7 different tribal backgrounds and from 8 
different native language backgrounds* All of the students included in the 
second year sample had also participated in the first year Y s data 
collection* The students in this sample are judged to be reasonably 
representative of Indian students attending rural schools on or near Indian 
-reservations- where the ■ Indian language and culture play a signif icant role 
in community life* 

In the first year of the study, two visits were made to most of the 
projects (only one visit was made to some very small projects)* The purpose 
of the first visit, in March of 1986, was to familiarize Title VII project 
staff, school principals, and other school staff members with the study, to 
compile rosters of the students to be included in the study, to identify the 
teachers and support staff members who work with these students, and, where 
required, to send home parent permission forms* Also during that visit, the 
study f s measure of academic aptitude (the Raven Progressive Matrices) was 
administered to students in the sample* All of the remaining study 
instruments (see Appendix C of Rudes et al* 5 1988, for a description of 
each) were administered during the second visit in April-June, 1986. 

During the spring of the 1986-87 school year, each of the projects in 
the study sample was visited one time* The purpose of the visit was to 
administer teacher questionnaires and to administer the study f s measures of 
academic achievement to sample students* (See Appendix B for a description 
of the study 9 s teacher and student instrumentation*) 
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1C 



C. ORGANIZATION OF THIS REPORT 



Chapter 2 of this report describes the characteristics of the students 
and the Instructional services which they receive, as well as the community 
and school context In which the services are provided. Chapter 3 examines 
the academic performance of the study students and Chapter 4 provides 
Interpretations and explanations for the findings presented in Chapter 3. 

Following these chapters are four appendices. Appendix A provides 
definitions for special terms and mathematical symbols used In this work. 
Appendix B provides the details of the study's sampling and data collection 
instruments and procedures. Appendix C provides more details regarding the 
analyses and interpretations in Chapter 4* and Appendix D provides the. names 
and affiliations of the members of the study's panel of technical advisors. 
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Chapter 2. DESCRIPTION OF SCHOOLS AND INSTRUCTIONAL PROGRAMS 1 



2A 



A, OVERVIEW 



This phase of the study encompasses Indian students in 11 isolated, rural 
elementary schools. The schools are in eight school districts located in six 
states. Two cohorts of students are involved. Students in Cohort A were 
enrolled in the first grade and students in Cohort B were in the third grade 
during the 1985-86 school year. As shown in Table 2.1 there were 278 students 
in Cohort A and 210 in Cohort B v with the numbers of students In the study 
ranging from a total of 117 students in the largest to a total of 32 students 
In the smallest of the school districts. Virtually all of the Indian students 
in each study school are from the same tribal group, with seven tribes 
represented overall. 



2B 



B. SCHOOL AND COMMUNITY SETTING 



2B.1 SCHOOL GOVERNANCE, LOCATION AND SIZE 



Schools attended by Indian students may be classified into various types 

according to the governing body responsible for school administration. This 

2 

study f s sample Includes 3 public schools, 4 tri bally controlled schools , 
and 1 Bureau of Indian Affairs (BIA) controlled school. 



Where a school Is located can be a useful Indicator of the socioeconomic 
status of students, and thus can be a barometer of schooling factors Including 



^Definitions of special terms and symbols used in this chapter are provided in 
Appendix A. 

^Tribally controlled schools are those schools which are operated by a tribe 
under contract to the Bureau of Indian Affairs. 
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the type of technological, curricular, and personnel resources available to 
students (Brookover & Schneider, 1975; Brookover & Lezotte, 1979; McDill & 
Rigsby, 1973)* There are two aspects of school location which are important 
in considering schools which serve Indian children. The first is whether or 
not the school is on or adjacent to a reservation. The second is the 
socioeconomic level of the immediate environs. In this study all of the 
schools are located on or adjacent to an Indian reservation; all are in 
relatively remote, rural areas; and as characterized by the study f s data 
collectors and by the principals of the study schools, all are in low-income 
areas. . 

School size has been found to affect how students are supported and 
challenged in the educational process (Flagg, 1964; Morocco, 1978) ♦ For 
Indian students this may be particularly, import ant. because it may influence_„ 
their fluency in English and how comfortable they feel in the academic program 
being provided in school, both being factors that may ultimately Influence 
student achievement. Equally important may be the proportion of Indian 
students in the student body. 

For the elementary schools that participated in the study, the number of 
students in grades K-6 averaged 186, with a range from 55 to 406. In six of 
the eleven schools, 100% of the students were Indian and in all cases Indians 
made up over 80% of the student body. An overview of the schools in the study 
in terms of grade K-6 enrollment and the percentage of Indian students is 
given in Table 2.2. 

2B.2 COMMUNITY, SCHOOL AND HOME LANGUAGE USE 



The extent of use of a local tribal language, English, and other languages 
varies greatly among Indian communities. There are whole communities which 
are nearly monolingual in an Indian language (e.g., some Isolated Navajo 
communities); communities where just about everyone is proficient in English 
and the native language, and in which people use both on a dally basis 
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TABLE 2.1. Characteristics of participating school districts 


)istrict 


Number of Schools 


No. Students In Study 




Tribal 


Code 


in Study 

" . . — - ■ 


Cohort A 


Cohort B 


Governance 


group 




1 


52 (19%) 


39 (19%) 


BIA 


Navajo 


910 


2 


61 (22%) 


56 (27%) 


Public 


Navajo 


903 


1 


24 (9%) 


23 (11%) 


Tribal 


Hualanai 


906 


1 


20 (7%) 


12 (6%) 


Tribal 


Apache 


912 


1 


30 (11%) 


18 (9%) 


Public 


Crow 


915 


1 


31 (11%) 


28 (13%) 


■Public 


Metchif 


920 


1 


24 (9%) 


10 (5%) 


Tribal 


L&kota 


926 


_3 


36 (13%) 


24 (11%) 


Tribal 


Choctaw 


TOTAL 


11 


278 (100%) 


210 (100%) 









TABLE 2.2. 


School enrollment in grades K-6 




School Code 


Total Enrollment 


Percent Indian 


Percent Indian 


Percent Indian 


Grade K-6 


Grade K-6 


Grade 1 


Grade 3 


9021 


368 


100% 


100% 


100% 


9101 


406 


89 


90 


87 


9102 


144 


85 


83 


83 


9031 


156 


83 


100 


100 


9062 


112 


100 


100 


100 


9121 


218 


99 


95 


97 


9151 


290 


100 


100 


100 


9201 


142 


100 


100 


100 


9261 


55 


100 


100 


100 


9262 


96 


100 


100 


100 


9263 


55 


89 


88 


73 



(e.g., some Crow communities); and communities which are nearly monolingual in 
English (e.g., some Indian communities in the eastern United States). Also, 
the extent of use of an Indian language, English, and other languages will 
vary from home to home within a community. 

a. Community Language Use 

To assess the extent to which Indian languages and English are used as 
means of dally communication in the study 1 s communities, data were collected 
from community leaders, and a sample of parents. They were asked to identify 
the language most frequently used in ten types of social situations. 1 For 
each of these situations, respondents were asked to specify whether the 
language or languages most frequently heard were: (1) the local Indian 
language; (2) English; or (3) another language. Respondents were also given 
the opportunity to state f Don't Know 1 for each situation. 

To arrive at an index of the extent of native language use in the 
communities associated with each sample site, a difference score was 
calculated by eliminating all 1 Don't Know 1 or f Other Language 1 responses, and 
subtracting the total number of positive responses to f Use English 1 from the 
total number of positive responses to f Use Local Indian Language f for all 
respondents from each community. The scores could range anywhere along a 
scale from -10 (English is the only language heard in all of the specified 
social situations) to +10 (the local Indian language is the only language 
heard in all of the specified social situations), with a score of zero 
indicating that English and the local Indian language are used about equally 
in the community. To determine the extent of native language use, data from 
all of the completed forms for a site were averaged to compute the score, 
since it was assumed that all of the students at a site came from a single 
community. 



l-The selection of specific social situations was taken from the linguistics and 
anthropological literature on language death and obsolescence (e.g., Dressier and 
Wodak-Leodolter, 1977). See Rudes et al . (1988 page D-7) for a listing of the 10 
specific situations. 
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The eight study communities represent a broad range of different situations 
with regard to use of English and the local Indian language. The project 
receiving the lowest rating received a score of - 7.60 (Project 915), 
indicating that. English was used predominantly in the community, but that 
there was at least some use of the Indian language as veil. Thus there was at 
least some — even though in some cases very little — use of the local Indian 
language In all of the project communities. Conversely, in the communities 
with scores indicating more use of the Indian language than of English, the 
highest score was + 503 (for Project 926), indicating some consistent use of 
English in all cases vsee Table 2.4). 

Because the extent of Indian language use in an Indian child f s community is 
potentially an Important variable and may be highly correlated with program 
services and their outcomes in terms of school performance in English, 
projects were put Into three categories for some subsequent analyses. These 
three categories are: (1) predominant use of the Indian language in the 
community: possible score range of +2.5 to +10; (2) roughly equal use of the 
Indian language and English: possible score range of -2.499 to +2.499; and 
(3) predominant use of English in the community: possible score range of -2.5 
to -10. The projects in each category are shown In Table 2.3, below. 



TABLE 2.3. Categories of community language use 
based on community language use index 



Category N 

(1) Predominant Use of English 1 
Language in the Community. 
Possible Range: -2.5 to -10 



(2) Roughly Equal Use of the 3 
Indian Language and English. 
Possible Range: -2.49 to +2.49 

(3) Predominant Use of the 4 
Indian Language In the 
Community. Possible Range: 

+2.5 to +10 



Project 
915 



903,906,920 



902,910,912 
926 



9 

ERJp 
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b. Language Use in the Hose 



More directly related to the child's language learning than the extent of 
use of English and the Indian language in the community is the pattern of 
language use found in the home. Parents who do not speak Ecglish in the home 
do not reinforce English skills learned in school, and may net be able to help 
with homework* 

Therefore, parents were asked which languages were used by the mother or 
female guardian In the home, and by the father or male guardian* The 
responses were combined to create three categories of language use by 
parents; (1) one or more non-English languages, but not English; (2) English 
and at least one other language; and (3) English only* 

Table 2.4 provides a project-by-project comparison of the pattern of use of 
English and an Indian language In students 9 homes with the index of Indian 
language use in the communities served by the projects* As the data in this 
table show, Indian language use in students f homes tends to be greater In 
communities where there Is greater use in general of the Indian language* 



TABLE 2*4* Home and community language use In 
participating districts 



Languages Used at Home by Indian Parents 





(Percentage of Parents) 






Index of Extent of 




Indian Language Indian Language English 




Indian Language Use 


Project 


Only 


and English 


Only 


N 


In the Community 


926 


72 


26 


2 


58 


+5.03 


902 


16 


74 


10 


19 


+4.70 


910 


3 


89 


8 


93 


+3.33 


912 








0* 


+2.50* 


906 


46 


36 


18 


22 


+1.00 


903 


2 


32 


16 


50 


+0.71 


920 


4 


85 


11 


27 


-1.00 


915 


0 


2 


98 


51 


-7.60 



*No parent questionnaires were completed at this site, and the index of 
community language use was based on data provided by school personnel and 
linguists familiar with the project* 
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c. Language Use in the School 

Because Indian students 1 English language proficiency is an Important 
factor in their educational attainment (Galliland, 1986), variables that 
affect the overall school language environment were identified. They include 
school district policies relative to the use of English and other languages 
both within and outside the instructional context, the principals f language 
background, the principals 9 attitudes toward the use of non-English languages 
in the school, and the use of English and other languages outside the 
classroom by principals, teachers, and students. 

School district policy toward the use of non-English languages provides a 
measure of the specialized resources available to assist Indian children, in 
the educational process, k specific district policy can also influence how 
instructional staff interact and react to Indian students. 

The data related to school policy toward the use of languages other than 
English are taken from responses to questions in the School District P alley 
Questionnaire. In 63% of the districts, respondents indicated that thare ?as 
a district policy concerning the teaching of languages other than English as a 
subject area in the elementary grades; Indian students may receive instruction 
in the oral and/or written language arts of their native language in all of 
the schools with an explicit policy. 

The extent to which Indian students, teachers, and other staff actually use 
a language other than English in non-instructional situations is another 
useful indicator of school language environment, A composite variable was 
therefore created that describes the extent to which principals, teachers, and 
students use a language other than English outside the classroom context. The 
composite is composed of three items: (1) the extent to which teachers use 
non-English languages when interacting with Indian LEP students; (2) the 
extent to which Indian LEP and English-proficient students use English when 
interacting outside the classroom; and (3) principals 9 use of a language other 
than English with non-English speaking students. A higher genre on this 
composite indicates greater use of a language other than English, 3.0 being 



the lowest score possible and 6.0 being the highest score possible. The 
distribution of this composite, as shown in Table 2.5, indicates that English 
is the primary language used in the large majority of schools. 

Taken together these findings reveal a change in school climate with 
respect to Indian language use from that reported as late as 1969 in "The 
Kennedy Report. 99 As indicated in that report and other references, the 
general policy of most schools, in particular Bureau of Indian Affairs 
schools, up to the 1970s was to discourage use of an Indian language by 
students, and even to punish them for its use. 



TABLE 2.5. Extent of non-English language use outside the 
classroom by principals, teachers, and students 



Composite* 
Score 


Number of 
Districts 


Percentage 


3.0-4.0 


5 


46% 


4.1-5.0 


3 


27 


5.1-6.0 


1 


9 


Hissing 


2 


18 


TOTAL 


11 


100X 



*The composite variable was created by combining responses from the three 
items: (1) The extent to which teachers use non-English languages when 
interacting with Indian LEP students; (2) the extent to which Indian students 
use English when interacting outside the classroom; and (3) principals 9 use of 
a language other than English with non-English speaking students. A higher 
score Indicates greater non-English language use, 3.0 being the lowest score 
possible and 6.0 being the highest score possible. 
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C. TEACHERS AND INSTRUCTIONAL PROGRAM 



During visits to the project schools each teacher of each student in the 
study was interviewed. Utilizing a specially developed questionnaire and 
reporting f orm* these teachers provided a description of the instructional 
program of each of their Indian students* Subsequently, the responses from 
each of the student's teachers were aggregated to provide a separate 
description of the instructional program of e • student in the study* 

Students in Cohort A received about 23 hours a week of academic Instruction 
and students In Cohort B received about 26 hours* Table 2*6 presents the data 
on the average number of hours per week of instruction in academic subjects 
for the Indian students* To help put the data for Indian students In this 
study into perspective, the table also shows the data from the 23 school 
districts in the first year of, the study* In overall total hours of 
Instruction, the Indian students in this study received more hours of academic 
instruction during Tear 1 than did the larger sample of Indian students* 

As the table shows, the major portion of the Instruction for this study f s 
students was In English language arts (approximately 58% of the weekly hours 
received by students in Cohort A and 47% of the hours received by students in 
Cohort B)* For most students, English language arts instruction was divided 
between the regular school curriculum and special supplementary instruction In 
English language skills* Special Instruction In English was received In both 
years of the study by 71% of the students in Cohort A and 43% of the students 
in Cohort B, with 73% of the Cohort B and 77% of the Cohort A students 
receiving such instruction during the study's initial year* 

While about half of the instructional time for both cohorts was devoted to 
English language arts, overall the students received an average of less than 
two hours a week of Instruction In the language arts of an Indian Language; 
and, as shown In Table 2*7, most of the students received no such Instruction 
at all* Indeed, only 20% of the students In Cohort A and 27% of 
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TABLE 2.6. Mean number of hours per week of instruction in all 
academic subjects for Indian students 



Cohort A Cohort B Indian Students in 

23 Year 1 School 
Districts* 

Year 1 Year 2 Year 1 Year 2 Cohort A Cohort B 

Subjects ; 
Regular English* * 



Reading 
Other " 
Regular English Total 


6.7 
4.0 
(10.7) 


5.8 
4.6 
(10.4) 


5.7 
3.8 
(9.5) 


5.9 
3.9 
(9.8) 


5.8 
4.0 
(9.8) 


■ 4.3 
3.7 
(8.0) 


Special English 0 














Oral English 
Other 

Special English Total 


3.3 
0 

(3.3) 


1.8 
0.6 
(2.4) 


2.6 
0 

(2.6) 


1.5 
0.7 
(2.2) 


2.0 
1.0 
(3.0) 


1.6 
1.3 
(2.9) 


Indian Language 
Reading 
Oral 

Indian Language Total 


0.7 
1.3 
(2.0) 


.4 
0.3 
(0.7) 


0.5 
1.0 
(1.5) 


0.8 
0.9 
(1.7) 


0.4 
0.9 
(1.3) 


0.3 
0.6 
(0.9) 


Mathematics 


4.1 


4.3 


5.2 


5.2 


4.0 


4.7 


Science 


1.6 


1.8 


■ 2.8 


2.6 


1.7 


2.3 


Social Studies 


1.3 


1.7 


2.7 


3.5 


1.5 


2.4 


Ethnic Heritage 


0.8 


0.8 


1.3 


1.2 


0.6 


0.9 


Total 


(23.8) 


(22.1) 


(25.6) 


(26.2) 


(21.9) 


(22.1) 


No. of Students 


250 


225 


185 


201 


576- 
577 


472- 
474 ■ 



a Indlan students In the 23 school districts Included In the first year of the 
study. The means are based on all students for whom data were obtained; when 
students did not receive Instruction in a particular subject area, a value of 
zero hours was Included In the mean. 

^"Regular English" refers to the English instruction provided to monolingual, 
English-speaking students and other students who are proficient in English. 
"Special English" refers to an Instructional program, such as ESL, that utilizes 
materials and methods especially designed for teaching English to LEP students. 
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TABLE 2.7. Percentage of students receiving instruction in language 
arts of an Indian language and the mean number of 
hours they receive 




Cohort A 


Cohort B 




Percent receiving 

instruction Mean hours 


Percent receiving 

instruction Mean hours 


Year 1 


46% 3.7 


36% 1.9 


Year 2 


26% 2.7 


30% 4.3 



the students in Cohort B received such instruction in both years of the study, 
and in only two of the districts did all of the students receive instruction 
in an Indian language. 

Thus, not surprisingly, almost all the instruction offered these students 
was provided in English. Indeed, in one project teachers made no use of 
Indian language when providing instruction. Across the rest, less than 20% of 
the instruction in Math, Science and Social Studies in either cohort in either 
year was provided in an Indian language. In Cohort A the average use went 
from 17% in year 1 to 12% in year 2, and in Cohort B it went from 18% to 11%* 
On a project-by-project basis, the amount of Indian language use in academic 
instruction decreased each year as well. For example, in the district where 
use of an Indian language during academic instruction was greatest, students 
in Cohort A received 40% of their instruction in math, science *nd social 
studies in their Indian language in year 1. In year 2 the mean for these 
students dropped to 28%. Third-graders in that district received 15% of their 
instruction in an Indian language, and in the fourth grade such instruction 
amounted to only four percent. 

Language of instruction is closely related to characteristics of the 
instructional staff, and whether or not Indian students* teachers are 
themselves Indian may affect students 9 interest and performance in school. 
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As shown in Table 2.8, 44% of students In Cohort A and 62% of students in 
Cohort B had an Indian as their main classroom teacher in at least one of the 
two years of the study* Although there is considerable variation in this 
regard wittiu cohort across the «ight districts, at least some students in all 
of the districts had at least one Indian teacher, thus indicating the presence 
of Indian professionals on the teaching staff of each of the study schools* 
Similarly, the table shows that most students — and at least some in every 
district — had at least one teacher who used the local Indian language when 
communicating with students outside of class* 

Whether students receive most of their academic instruction from a 
professional teacher as opposed to a classroom aide may also Influence student 
performance* Presumably it is preferable for students to be Instructed by 
certified teachers, but often students with limited proficiency in English 
receive substantial portions of their instruction from an aide* To determine 
the extent to which this was true in this study's sample the main classroom 
teachers of each of the students were asked to indicate whether "when Native 
American students receive Instruction In areas such as math, science and 
social studies ••• (a) the classroom teacher provides most/all; (b) the aide 
provides most/all, or (c) both teacher and aide share this instruction 
equally * w Teachers- were also given the opportunity to indicate none of the 
above was the case* As shown in Table 2*9, a substantial portion of the 
academic instruction provided to a large proportion of the students, 
particularly in year 1, was provided by an aide, and in two of the districts 
an aide provided all of the instruction to the first graders in the study's 
initial year. 
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TABLE 2.8. 



I1C 



Percentage of students "whose main teacher Is Indian 
and percentage with a teacher who used an Indian 
language with students outside of class 



Percentage of students 
with a teacher In Tear 1 
or 2 who used an Indian 
Percentage of students 1 whose main teacher Is Indian language with students 
Cohort A Cohort B 



District 






Year 1 






Year 1 


Cohort 


Coho 


Code 


Year 1 


Year 2 


or 2 


Year 1 


Year 2 


or 2 


a 


B 


902 


33% 


64Z 


62Z 


53Z 


0Z 


53Z 


94Z 


47Z 


910 


51 


35 


35 


68 


34 


50 


35 


55 


903 


4 


76 


68 


0 


100 


100 


88 


100 


906 


0 




0 


100 


100 


100 


0 


100 


912 


100 


27 


100 


94 


39 


94 


70 


94 


915 


0 


0 


0 


61 


46 


75 


0 


57 


920 


0 


100 


83 




0 


0 


100 


0 


926 


0 


0 


0 


38 


47 


38 


8 


42 




27 


41 


44 


55 


43 


67 


51 


61 



TABLE 2.9. Percent of students who received a substantial amount of 
academic Instruction from a classroom aide 



District 


Cohort 


A 




Cohort 


B 


Code 


Year 1 


Year 2 


Year 


1 


Year 2 


902 


63. 5Z 


100Z 


OZ 




OZ 


910 


100* 


32* 


76 




32 


903 


100* 


0 


100 




0 


906 


0 




0 




0 


912 


50* 


0 


6* 




0 


915 


100 


0 


0 




0 


920 


0 


100* 


0 




0 


926 


0 


100 


0 




0 


TOTAL 


51Z 


47% 


36Z 




185 



*Mo8t or all is from an aide. 
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Chapter 3. CHARACTERISTICS AND PERFORMANCE OF INDIAN STUDENTS 



A. STUDENT CHARACTERISTICS 



The students in the study represent six Indian tribes and virtually all 
(99%) were born on or near an Indian reservation. Residence on or near a 
reservation is an important variable in understanding Indian student s f 
academic performance because of the important cultural and linguistic 
influences on Indian children of life on the reservation. For most students, 
their current school was the only school they had ever attended. Almost all 
(96%) had attended kindergarten and 76% had attended at least one year of a 
pre-school program. The mean age for first-grade students in the study (as of 
January 15, 1986) was 7.1 years (S.D.-.52) and 9.2 years (S.D.-.65) for 
third-grade students. These are in the range of the ages one would expect for 
first and third-grade students. 

3A.1 ORAL LANGUAGE PROFICIENCY 



To get a uniform measure of English and Indian-language proficiency for 

students across all projects in the study, arrangements were made for teachers 

of students in the study sample to complete an instrument entitled the Student 

2 

Oral Proficiency Rating (SOPR). On this instrument, students were rated by 



^Definitions of special terms and symbols used in this chapter are provided in 
Appendix A. 

2 The SOPR is a rating instrument that is a slightly modified form of the Student 
Oral Language Observation Matrix (SOLOM), an instrument used in California to 
assist in student placements. The SOPR provides a measure of student proficiency 
based on actual comprehension and production within formal and informal classroom 
discourse situations. The data that form the basis of the teacher ratings of 
student oral proficiency are the numerous classroom discourse situations in which 
the teacher and the student have used the language of interest. Thus the data 
used are drawn from extensive daily interaction with the student and are not 
limited only to selected topic areas or selected language skills. Since no 
specific assessment situation is required for the rating, student reticence or 

. test-wiseness is much less a factor in the ratings. For these reasons, the SOPR 
ratings are expected to be more valid for the study purposes than any scores 
obtained through the use of the tests available commercially. Also, the general 
format of the SOPR is such that it can be used for all language groups, provided 
that there is a qualified fcf \cher available to rate the student in the native 
language. (For additional discussion see Rudes et al. , 1988, and Zehler, 1985.) 
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teachers who were proficient in the language being rated, whether English or 
the Indian language, and who were also familiar with the student 9 8 use of that 
language within a range of classroom situations. Students were rated on a 
scale of 1 to 5 in five categories of oral proficiency: comprehension, 
fluency, vocabulary, pronunciation, and grammar. A rating of 1 indicated 
minimal or no proficiency in that category of language proficiency while a 
rating of 5 indicated ability equivalent to that of a monolingual speaker of 
the same age as the student being rated. A total score was calculated by 
summing the scores for the five individual categories; the total score 
possible thus ranged from 5 to 25 with the exception that for all the students 
in one site who were known never to have heard an Indian language the total 
score of 5 was changed to 0 (or to 4 for use in some analyses). 

The means of the total scores for the English and Indian language ratings 
are presented in Table 3.1. As the table shows, the language proficiency of 
the students varies by district, there being greater variation among districts 
with respect to the Indian language than English. Overall, the students may 
be considered as having a functional, but not fully fluent, proficiency in 
English which was a little greater for third-graders than first-graders. 
Proficiency in an Indian language is rather limited for both first and 
third-graders, with students in one of the projects having no proficiency at 
all. 



TABLE 3.1. Mean English and Indian language SOPR scores 



Dis- 
trict 
Code 



lean 



Cohort A 



English 



SD 



No. 



Cohort B 



Mean SD 



No. 



Indian Language 



Cohort A 



Mean SD 



No. 



Cohort B 



Mean 



SD 



No 



902 
910 
903 
906 
912 
915 
920 
926 
TOTAL 



17.7 
19.8 
19.3 
15.6 
20.6 
21.2 
20.6 
17.2 
19.0 



5.16 
2.68 
2.33 
3.72 
4.74 
3.49 
2.36 
3.81 
4.15 



51 
39 
24 
16 
30 
28 
24 
36 
248 



17.1 
19.2 
20.6 
19.2 
22.2 
20.4 
24.6 
17.4 
19.5 



6.01 
3.00 
2.25 
2.44 
3.60 
2.63 
.52 
4.20 
4.26 



38 
40 
23 
9 
16 
28 
10 
24 
188 



16.8 
15.5 
12.5 
17.8 
14.8 
4.0 
14.8 
17.5 
14.4 



5.88 
4.50 
4.67 
7.14 
8.55 
0 

4.92 
4.84 
6.78 



47 
39 
24 
16 
30 
29 
24 
36 
245 



18.1 
17.2 
16.3 
12.4 
18.7 
4.0 
14.8 
19.8 
16.3 



6.54 
6.17 
5.70 
7.48 
6.35 
0 

6.97 
3.80 
7.00 



39 
40 
23 
7 
16 
15 
1 

24 
174 
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3A.2 ACADEMIC APTITUDE 



To provide a measure of the child 1 8 academic ability which would not be 
operationally dependent on a knowledge of the English language the Raven 
Progressive Matrices Test was used. A review of the distribution of total 
scores on the Raven for grades 1 and 3 indicates that a very large part of the 
entire range of possible scores is covered • When scores on the Raven are 
compared to national percentiles for a cross section of students of the same 
age in the US, the Indian students in the sample score at about the national 
norm. As shown In Table 3.2, the mean score for the students in Cohort A Is 
equivalent to the 58th percentile and the mean score for Cohort B is 
equivalent to the 45th percentile (Raven et al M 1986). As the table also 
shows, there is some variation among the sites. Relatively low mean scores 
for students in both cohorts are found at site 903 and 920. Similarly, the 
mean score Is quite low for cohort B students at site 906. While we do not 
have the data to clearly explain the reason for these particularly low means, 
we suspect It is related to the fact that the communities served by these 
sites are all extremely Isolated, traditional, and poor, even in comparison to 
the other sites in this study, and that in district 906 students may leave the 
tribally controlled study school for the nearby public school after they have 
increased their English language ability. 



TABLE 3.2. Mean Raven scores and comparison of scores 
with the publisher 9 8 national sample 



Dis- 




Cohort 


A 








Cohort 


B 






trict 


Mean 






Mean 




Mean 






Mean 


% 


Code 


Score 


SD 


No. 


Age 


%ile 


Score 


SD 


No. 


A*e 


lie 


902 


19.8 


5.27 


50 


7yr2mo. 


59 


29.7 


7.84 


35 


9yr3mo. 


42 


910 


20.8 


4.86 


58 


7-1 


64 


25.6 


3.36 


48 


9-1 


40 


903 


16.6 


4.85 


24 


7-3 


35 


24.5 


8.05 


22 


9-4 


27 


906 


21.2 


6.11 


16 


6-11 


66 


26.5 


9.74 


8 


9-4 


33 


912 


23.5 


5.33 


30 


7-1 


77 


32.7 


7.28 


15 


9-2 


62 


915 


19.6 


4.61 


29 


7-1 


58 


27.6 


7.61 


27 


9-1 


43 


920 


15.3 


6.36 


24 


7-2 


35 


27.5 


9.18 


10 


9-8 


36 


926 


19.0 


5.32 


34 


7-2 


55 


25.8 


9.08 


24 


9-0 


40 


TOTAL 


19.7 


5.63 


265 


7-1 


58 


27.2 


8.18 


189 


9-2 


45 




35 
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Not surprisingly, further analyses indicates that Raven scores definitely 
are related to mastery of English. The correlation between Raven total and 
English SOPR is .17 for grade 1 and .30 for grade 3; these correlations are 
not high, but they are statistically significant* Bearing in mind that the 
Raven is a nonlanguage test, it seems unlikely that mastery of English is 
among the causal factors accounting for high Raven scores. Rather, we 
surmise, there is some causality in the opposite direction; students with high 
Raven scores tend to have a high level of academic aptitude including the kind 
. of verbal ability that is useful in learning English. There appears to be no 
such causal relation for the Indian-language SOPR, however. The explanation 
is probably that all the students, regardless of the extent to which they have 
been exposed to their tribe's Indian language, either in or outside of school, 
are under some pressure from the school to acquire proficiency in English (or 
if they are already proficient in English to continue improving their skills 
in it). Conversely, there is not likely to be uniform pressure on students to 
augment their skills in the tribal language. 

3A.3 HOME ENVIRONMENT 

Family structure and parents 9 educational levels are also factors which 
have been shown to be related to academic achievement (Laosa, 1982a; Laosa, 
1982b; Henderson, 1981; Lambert, 1977; National Center for Education 
Statistics, 1978; Rosenthal, Baker & Ginsberg, 1983). There is also 
considerable evidence to suggest that parents ' interest and involvement in 
education can affect the academic outcomes of their children (Gore, 1974; 
Kjolseth, 1972; Cervantes, 1978; Cervantes, Baca, & Torres, 1979). Therefore, 
a series of questions was asked relating to these areas. 

The children in the study came from moderately large families. Overall, 
the mean family size of students in the study (both Cohort A and B) was 5.9 
members, with only moderate variation among school districts (from 7.8 to J.l 
members). With respect to education, the fathers of Indian children had 
completed about a half year more of schooling than mothers, but the mean for 
neither was above the 11th grade (i.e., 10.17 years for mothers and 10.75 for 
fathers). Again, there was relatively little variation across communities 
with respect the the years of schooling of either mothers or fathers of the 
Indian students in the study. - q£ 
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The academic orientation of a family can also be Inferred from various 
activities in a household. Parents with high educational expectations may 
require that children spend more time on homework, may read more to their 
children, or may encourage more reading* Table 3.3 shows the mean number of 
hours per week which Indian parents reported that their children spent doing 
homework, reading (other than homework), and being read to. As is shown, on 
the average these Indian students spend a little over three hours a week on 
homework, but there is considerable variation from site to site. 

Parents were also asked to estimate the amount of time their children 
watched TV or VCR programs In English, listened to the radio In English and 
listened to the radio In another language. As shown in Table 3.4 there was 
considerable difference between districts but little difference across the two 
cohorts. Total listening and viewing time equaled an average of 12.3 hours 
per week. 







TABLE 


3.3. 


Mean hours per week spent by Indian students doing 












homework, reading (other than homework), and- being read to 




Dis- 




Doing 


Homework 


Read! 


Lng 


Being Read to 


trict 


Cohort A 


Cohort B 


Cohort A 


Cohort 


B 


Cohort A 


Cohort B 




Code 


Hrs 


N 


Hrs 


N 


Hrs 


N 


Hrs 


N 


Hrs 


N 


Hrs 


N 


902 


1.5 


8 


2.8 


8 


1.9 


8 


1.5 


8 


1.0 


8 


1.1 


8 


910 


3.2 


34 


3.1 


40 


1.9 


34 


2.6 


40 


2.0 


34 


1.6 


40 


903 


4.5 


24 


3.8 


23 


3.8 


24 


2.9 


23 


3.5 


24 


2.4 


23 


906 


1.5 


13 


3.0 


5 


.9 


13 


1.0 


5 


1.0 


13 


.2 


5 


912 


NA 




NA 




NA 




NA 




NA 




NA 




915 


1.6 


20 


3.1 


19 


* 2.6 


20 


3.1 


19 


2.4 


20 


.7 


19 


920 


1.9 


17 


2.3 


9 


3.1 


17 


1.2 


9 


2.8 


17 


2.1 


9 


926 


4.7 


30 


5.0 


22 


1.5 


30 


1.6 


22 


1.0 


30 


.6 


22 


TOTAL 


3.1 


146 


3.5 


126 


2.3 


146 


2.3 


126 


2.0 


14-5 


1.4 


126 



37 
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TABLE 3.4. Mean hours per week spent by Indian students 

watching TV or VCR programs in English, listening to 
radio programs in English, and listening to radio 
programs in another language 



Cohort A Cohort B 



District 


TV/ 


English 


Other 




TV/ 


English 


Other 




Code 


VCR 


Radio 


Radio 


N 


VCR 


Radio 


Radio 


N 


901 


4.1 


1.4 


, .4 


8 


7.1 


3.6 


1.6 


8 


910 


8.4 


2.9 


.8 


34 


5.2 


2.3 


.7 


40 


903 


9.9 


3.1 


.3 


24 


10.4 


2.4 


.1 


23 


906 


2.4 


.4 


0 


13 


5.4 


1.8 


0 


5 


912 


NA 


NA 


NA 




NA 


NA 


NA 




915 


11.8 


3.3 


0 


20 


15.1 


2.5 


.1 


19 


920 


9.9 


4.7 


1.4 


17 


7.4 


2.1 


1.0 


9 


926 


13.5 


3.1 


.1 


30 


11.6 


3.4 


.9 


22 


TOTAL 


9.6 


2.9 


.5 


146 


9.1 


2.6 


.4 


126 



The parents ' Interest in schooling is also reflected in their perceptions 
of the relative Importance of the education which the child receives In school 
versus the education which the child receives In the home, in the community, 
and elsewhere outside the school. When asked their opinion on this Issue, 94% 
of the parents stated that they considered the education their children 
receive in school to he "very important. w 

Parents* perceptions of the quality of the education their children 
received were somewhat lower than their assessment of its Importance. 
Overall, 13% rated the education their children were receiving as fair to 
poor, 32% as good, 30% as very good, and 25% as excellent. 

Finally, the academic orientation of a family may be reflected in the 
expectations which parents have for the amount of schooling which the child 
will probably complete. The parents of 45% of the Indian students expected 
their ctiild to go on to college, including 8% who expected their child to go 
to graduate school. Less than 3% of the parents expected their child to go to 
school through no more than the 9th grade. 
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B. ACADEMIC PERFORMANCE 



The vocabulary, reading comprehension, and mathematics subtests of the 
Stanford Achievement Test (SAT) were selected as the principa. measures of 
academic achievement for this study. 1 The overall study design called for 
testing students in the spring of 1986 and the spring of 1987, and the results 
of analyzing the two years of test data are presented below. 

A comparison of the Indian students with national norms is presented in 
summary form in Table 3.5, and in more detail in Tables 3.6 and 3.7. As the 
tables show, overall the performance of the Indian students is quite low on 
all of the tests, ranging from the 15th percentile to the 35th. Perhaps even 
more disturbing is the trend of the scores across the four grades. Figure 3.1 
graphically shows this trend. The scores for vocabulary, reading and math 
relative to national norms all decline sharply from the 1st to the 2nd grade. 
From the 3rd to the 4th grade there is neither a decline nor improvement with 
respect to vocabulary or math, and although there is a relative Improvement In 
reading In terms of the actual number of items correct the difference is an 
Improvement of less than two points (see Table 3.6b). 



TABLE 3.5. Summary of SAT percentile scores 
for vocabulary, reading and math 



Mean Percentile 



Cohort A 



Cohort B 



Year 1 



Year 2 



Year 1 



Year 2 



Vocabulary 
Reading 
Total Math 



26% 

28 

24 



15% 

20 

19 



19% 

25 

27 



8% 
35 
27 



39 
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trlet 
Coda 

902 
910 
903 
90ft 
912 
915 
920 
926 
TOTAL 



TABLE 3.4(a) SAT Vocabulary »nd rt*Hln« acoraa aft* tut tonal parcantilaat Cohort A 



H 

48 

39 
23 
20 
30 
30 
23 
33 
248 



16.5 
17.2 
19.1 
13.7 
22.5 
23.4 
23.1 
18.2 
19.2 



Vocabulary" 



CT til a 



4.32 
4.15 
3.77 
3.50 
3.49 
6.47 
3.39 
3.24 
5.60 



.16 
.19 
.26 
.13 
.41 
.46 
.44 
.22 
.26 



n 

47 
61 
24 
20 
21 
31 
20 
34 
1238 



SAT EaaUsh Scoraa 



Taar 2 



14.6 
16.0 
13.7 
13.7 
18.8 
16.7 
18.3 
14.7 
16.1 



CT Xlla 



3.03 
3.38 

2.13 
4.41 
6.00 
4.49 
3.63 
4.78 



.11 
.13 
.14 
. 8 
.26 
.26 
.25 
.11 
.13 



48 

39 

2'i 
2tf 
30 
30 
23 
33 
248 



faar 1 



16.3 
19.3 
22.4 
17.4 
28.5 
24.6 
29.4 
23.3 
22.2 



ftaadlng" 



C Xlla 



6.90 
5.50 
6.04 
6.56 
9.10 
9.16 
8.X6 
8.10 
8.92 



.13 
.22 
.28 
.16 
.44 
.35 
•46 
.31 
.28 



48 

60 
24 
20 
22 
31 
19 
34 
258 



16.6 

16.7 
17.8 
17.4 
27.5 
22.5 
24.6 
20.0 
19.5 



CP Xlla 



7.47 
6.83 
6.24 
7.87 
8.34 

£.46 
6.90 
8.53 



.15 
.13 
.17 
.16 
.39 
.26 
.30 
.21 
.20 



TABLE 3.6(H) SAT Vocabulary »nd madia* acoraa and national parcanttlaat 



Cohort B 



SAT Entllah Scoraa 



Taar 1 



Vocabulary' 



Taar 2 



Raadlr 



Yaarl 



Taar l 



Dla- 

trict 
Coda 




N 



35 
27 
22 
8 
6 

23 
10 
24 
160 



12.86 
16.15 
16.77 
12.50 
18.50 
19.04 
17.40 
11.96 
15.38 



<T Xlla 



C~ tlla 



cr tlla 



<T Xlla 



4.39 
6.21 
4.09 
4.87 
5.21 
6.30 
4.99 
4.10 
5.66 



.11 
.22 
.24 
.10 
.31 
.33 
.27 
. 9 
.19 



39 12.33 4.35 . 9 

55 18.84 8.61 .30 

23 15.74 4.53 .18 
.12 10.42 4.36 . 5 

16 16.31 4.44 .20 

28 19.64 5.83 .34 

9 16.22 5.45 .20 

24 13.33 5.04 .11 
206 15.93 6.74 .19 



35 
39 
22 
8 

17 
29 
10 
24 
183 



26.66 
24.08 
34.64 
23.25 
36.53 
30.46 
33.00 
26.29 
28.72 



9.30 
8.69 
9.14 
8.60 
10.13 
12.97 
12.72 
11.97 
ll.U 



.21 
.17 
.37 
.16 
.42 
,28 
.S3 
.21 
.25 



37 
56 
23 
12 
18 
28 
10 
24 
208 



26.84 
32.73 
31.26 
18.67 
32.89 
34.96 
32.40 
28.58 
30.53 



11.07 
12.99, 
10. f 
5.18 
11.60 
13.95 
10.79 
10.55 
12.18 



.27 
.39 
37 
.12 
.40 
.45 
.39 
.31 
.35 



.40 
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TABLE 3.7(«) SAT Hach scores and national percentiles: Cohort A 



Concept of Noeber 



Computation and Application 



Year 2 



fear t 



fear 



Year 1 



Dis-^ 

trict 

Coda 



<r xii« 



or ttle 



tile 

N/A 
N/A 
N/A 
N/A 
N/A 
K/A 
N/A 
N/A 
N/A 



902 

910 

903 • 

90S 

912 

915 

920 

925 

TOTAL 



4a 

39 
23 
20 
29 
30 
23 
35 
247 



18.17 
21.82 
U5i96 
17.80 
22.17 
22.20 
24.65 
18.29 
20.09. 



5.04 
4.57 
3.71 
6.21' 
5.06 
5.71 
4.01 
5.26 
5.57 



.17 
.30 

:io 
;15 
.32 
.32 
.43 

.17 
.22 



42 
58 
24 
20 
22 
31 
19 
34 
250 



16.95 
16.09 
15.88 
16.20 
19.23 
18.65 
22.58 
17.41 
17.49 



5.98 
4.73 
6.60 
5.67 
5.40 
7.15 
3.42 
8.97 
6.06 



.20 
.17 
.17 
.18 
.29 
.27 
.44 
.22 
.22 



48 

39 
23 
20 
30 
30 
23 
35 
248 



26. 44 
28.97 
22.35 
22.20 
25.23 
30.63 
30.48 
24.00 
26.51 



7.08 
6.99 
6.34 
8.38 
8.89 
6.53 
6.55 
9.41 
8.07 



.24 
.32 
.15 
.U 
.21 
.38 
.37 
.19 
.25 



36 
60 
24 
20 
22 
31 
20 
34 
247 



37.83 
37.07 
36.54 
35.30 
35.77 
41.84 
41.30 
32.12 
37.13 



11.69 
9.87 
15.34 
12.75 
9.11 
14.84 
8.86 
13.26 
12.21 



TABLE 3.7(a) SAT Math scores and national percentiles: Cohort A (Cont.) 



Total Math 



Tear I 



rear 



Dis- 
trict 
Code 



H 



Ule 



N 



tile 



902 



910 
903 
906 
912 
913 
920 
926 
TOTAL 



48 

39 
23 
20 
29 
30 
23, 
3f/ 
24> 



44.60 
50.80 
38.30 
40.00 
47.83 
52.83 
55.13 
42.29 
46.65 



11.48 
10.97 

9.14' 
14.21 
12.87 
11.33 

9.11 
13.66 
12.72 



.21 

.32 

.13 ' 

.15 

.27 

.36 

.40 

.18 

.24 



35 
57 
24 
20 
22 
31 
19 
34 
242 



55.97 66.19 
53.83 13.25 
52.42 21.35 
51.50 17.99 
55.00 13.33 
60.48 21.41 
63.16 11.33 
49.53 19.25 
54.89 17.25 



.19 
.17 
.15 
.15 
.19 
.25 
.29 
.13 
.19 



41 
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TABLE 3.7(b) SAT Hath scons sad nations! pscesntllss: Cohort B 





Concapt of Nuabtr 


Corporation 




Taar 1 


Taar 2 


Taar 1 


Taar 2 


Dis- 


































trict 


































Coda 




X 


o- 


Slli 


If 


7 


O* 


X1U 


if " 




cr 


SUA 


If 


*x 


or 


til*! 


902 


35 


18.09 


5.23 


.31 


37 


13.54 


5.01 


.24 


35 


25.74 


9.09 


.33 


37 


18.73 


8.71 


24 


910 


58 


20.03 


6.23 


.41 


55 


18.27 


7.51 


.44 


39 


24.08 


6.94 


.29 


56 


22.43 


7.74 


34 


90.1 


22 


18.82 


5.27 


•34 


23 


14.61 


3.94 


.29 


22 


25.77 


4.63 


.34 


1 


13.00 




U 


906 


8 


15.83 


8.98 


.23 


12 


11.33 


5.26 


•16 


8 


16.25 


10.93 


•12 


12 


10.92 


4.46- 


8 


912 


17 


20.82 


4.17 


•44 


IS 


16.44 


4.62 


.36 


17 


20.65 


6.76 


.21 


17 


24.41 


7.33 


40 


915 


28 


17.04 


4.93 


.27 


25 


16.11 


6.53 


.35 


28 


24.43 


7.95 


*30 


28 


21.54 


8.31 


32 


920 


10 


21.10 


4.01 


.45 


10 


17.50 


6.06 


.41 


10 


19.90 


7.46 


.19 


10 


25.70 


6.22 


43 


926 


24 


14.25 


6.26 


.19 


24 


13.79 


6.71 


.25 


24 


18.67 


9.30 


•16 


24 


17.75 


9.25 


22 


TOTAL 


182 


18.23 


5.90 


.32 


207 


15.61 


6.39 


.33 


183 


23.06 


8.25 


•26 


185 


20.51 


8.57 


29 



TABLE 3.7(b) SAT Math aeons and national pntcnntilns: Cohort B (Coat.) 





Hath Applications 


Total Hath 




Year I 


Tear Z 


Taar 1 


Taar 2 


Dis- 
































trict 
































Coda 


* 


X 


<r 


Tils 


M 


X 


<T 


tila 


N 


X 


& 


SUa 


If 


x O- 


Sill 


902 


35 


16.43 


6.72 


.24 


39 


16.67 


8.56 


.26 


35 


60.26 


18.00 


.29 


37 


48.22 19.37 


J21 


910 


38 


15.24 


5.58 


.20 


56 


16.98 


6.98 


.27 


38 


59.47 


13.79 


.28 


55 


57.93 19.23 


33 


903 


22 


20.32 


4.39 


.36 


23 


17.22 


5.88 


.27 


22 


64.91 


11.31 


.36 


1 


48.00 - 


.21 


906 


4 


9.50 


2.08 


. 7 


12 


10.58 


6.37 


.11 


4 


29.75 


12.69 


• 4 


12 


32.83 14.17 


.8 


912 


16 


20.69 


5.17 


.38 


16 


20.81 


7.64 


.37 


16 


63.69 


11.99 


.34 


15 


60.53 17.83 


36 


915 


28 


17.61 


7.05 


.28 


28 


23.18 


8*15 


.45 


28 


59*07 


16.07 


.27 


28 


60*82 20.22 


36 


920 


10 


19.50 


5.46 


.34 


10 


21.00 


6.58 


.38 


10 


60.50 


16.04 


.30 


10 


64.20 15.34 


AQ 


926 


24 


15*50 


8.58 


•21 


24 


15.87 


9.53 


.24 


24 


48.42 


22.64 


.16 


24 


47.42 23.99 


20 


TOTAL 


177 


17.12 


6.68 


.26 


208 


17.77 


8.13 


.29 


177 


58*51 


17.14 


.27 


182 


53.86 20.77 


27 
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National percentiles of mean SAT vocabulary, reading 

AND TOTAL HATH SCORES ACROSS THE FOUR GRADES OF THE STUDY 




V: vocabulary 

R: reading 

M: total math 

{concept of number, 
applications and ■ 
computation) 



2 3 4 

Grade 



43 
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The decline in SAT scores relative to norms differed for different levels 
of English oral proficiency* Table 3*8 presents the SAT means and the 
corresponding percentiles for students classified on the basis of English SOPR 
category. The "Gain" rows in this table show the year 2 minus year 1 
difference in percentiles. 

As Table 3.8 shows, the percentile differences for Cohort A are all 
negative; in other words there appears to be a systematic loss between grades 
1 and 2. Furthermore, the high SOPR category (23-25) generally shows larger 
declines in percentile than the two lower categories; this is probably partly 
due to regression towards the mean, resulting from the fact that the variable 
on the basis of which the students were categorized for this analysis — i.e. 
English SOPR — has a higher correlation with year 1 performance than with 
year 2. The correlations between SAT scores and English SOPR are shown in the 
last column of Table 3.8 to facilitate comparison. The largest percentile 
loss for the high SOPR group is 18 points (for Cohort A Vocabulary); this 
coincides with a sharp drop in correlation with SOPR from .60 to .46. However 
in view of the overall pattern of decline in percentiles from year to year it 
seems most unlikely that regression towards the mean is the sole explanation. 
The students who start out low in terms of SOPR also start low on the SAT, and 
they continue low. At best ther are barely holding their own; the high SOPR 
students, who do not have much of a deficit, if any, in English in year 1 
actually lose ground in year 2. For Cohort B (i.e., the change from grade 3 
to grade 4) the results are slightly better, in the sense that at all three 
SOPR levels the students seem to be improving in Reading Comprehension and 
holding their own in the other two subjects. The Cohort B year 1-to-year 2 
changes in terms of percentile level seem to be fairly uniform for the 
different SOPR levels. The reason for the difference between the results for 
Cohort 8 A and B is not evident; it may be due to chance. 

On a district by district basis the Cohort A results are no more 
encouraging. As shown in Table 3.9 which provides a summary of the individual 
district percentiles in vocabulary, reading and math across the two years, the 
percentiles corresponding to project means are low. Of the 96 means (8 
districts x 3 subtests x 2 years), 7% are in the 0-10 category and 38% are 



9 

ERLC 



44 



DEVELOPMENT ASSOCIATES* INC. 



3-13 



TABLE 3.8 SAX aasa rights scores sal corresponding percentiles for students 
la three categories in terms of English SOPE; also correlation 
between SAX score end English SOPX 




Eng.SOPR 
5-17 



Bag.SOR 
18-22 . 



Eng.SQPa 

23-25 







M 




* 


K 


Xile 


H 


M 


Ule 


S 


Eng.SOP* 


A 


1 1 


15.1 


11 


65 


19.4 


27 


110 


24.1 


49 


50 


. .60 




2 2 


13.3 


8 


65 


16.0 


15 


110 


19.7 


31 


50 


.46 




Gain 




-3 






-12 






-18 






B 


1 3 


U.3 


8 


35 


15.7 


20 


85 


17.9 


29 


36 


.42 




2 4 


li.8 


8 


35 


15.6 


18 


85 


18.3 


28 


36 


.40 




Gala 




0 






-2 






-1 






A 


1 1 


16.0 


12 


65 


22.7 


29 


109 


28.9 


45 


50 


•50 




2 2 


14.2 


10 


65 


20.7 


22 


109 


25,7 


33 


50 


.49 


t 


Gain 




-2 






-7 






-12 






B 


1 3 


19.1 


10 


40 


28.4 


24 


94 


37.7 


44 


45 


.56 




2 4 


22.1 


18 


40 


30.1 


34 


94 


38.7 


53 


45 


.48 




Gain 




8 






10 






9 






A 


1 1 


38.0 


13 


57 


48.5 


28 


102 


56.1 


43 


49 


.56 




2 2 


42.2 


7 


57 


56.7 


19 


102 


65.7 


33 


49 


.45 




Gain 




-6 






-9 






-10 






B 


1 3 


45.4 


13 


37 


57.6 


26 


77 


67.8 


40 


35 


.49 




2 4 


42.1 


15 


37 


53.6 


27 


77 


66.9 


44 


35 


.46 




Gala 




2 






1 






4 
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i;\ below the 20th percentile In terms of national norms. This Is a poor shoving 

' In view of the fact that school means are expected to cluster more tightly 

around the overall mean than do Individual students. How much of the sub-par 
performance in math is due to deficits in the English language skills 
r.. (vocabulary and reading) is not entirely clear; but what is clear Is that 

f progress in reducing those deficits (with the possible exception of reading 

skills for Cohort B) is discouragingly slow. 





TABLE 3.9. Number of projects in various SAT 

percentile categories for vocabulary, 
reading and math 


Percentile 


Vocabulary 


Cohort A 

Reading Total Math 


Cohort B 
Vocabulary Reading Total Hath 


0-10 


1 


0 0 


4 0 2 


11-19 


. 7 


6 9 


3 3 1 


20-29 


5. 


4' 4 


5 4 6 


30-39 


0 


4 2 


4 6 6 


40-49 


3 


2 1 


0 3 1 


'Categories combine the two years of the study; each project was counted twice 
for each subtest (i.e., Tear 1 and Tear 2) 
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Chapter 4. POSSIBLE EXPLANATIONS FOR INDIAN STUDENTS' 
LOW ACHIEVEMENT TEST SCORES 



4A A. INTRODUCTION 

The major finding from the previous chapters is that the academic 
achievement test scores of the Indian students are extremely low and that over 
the two years of the study they either got worse or stayed the same* This is 
in spite of the fact that these students have academic aptitudes equivalent to 
published national norms and are attending schools which receive federal 
grants to provide special, language-related instructional services to these 
students* While these results are substantially worse than results from the 
most recent major study of Indian students in schools receiving funds through 
the federal Indian Education Act (Young et al. , 1983) , the two samples are 
quite different. AH the students in this study were residents of relatively 
traditional. Isolated communities on or near reservations, while the earlier 
study Included many Indian students living In urban and suburban locales* 

The purpose of this chapter is to report on various analyses undertaken to 
explain, or at least suggest reasons for the low scores* The test results for 
these students In the first year of the study are essentially the same as 
those for the larger sample of Indian students of which they are a part (Rudes 
et al * , 1988) , and there is no reason to believe the second year results for 
the larger group would not have been similar as well. Thus, the data in this 
study suggest that Indian students have serious academic deficiencies which 
their schools do not seem to correct* Clearly a serious problem exists, and, 
although beyond the original purpose of this study, it Is Important to use the 
data at hand to better understand the nature of that problem and to provide 
direction to future inquiries* 
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4B B. SUMMARY OP EXPLANATORY ANALYSES 



In order to gain additional insights concerning the Stanford Achievement 
Test results, a series of descriptive and inferential statistical analyses 
were performed* These analyses fell into four basic categories v 



• multiple regression analyses, in which various combinations of variables 
were used to predict outcomes on Year 1 and Year 2 SAT scores in 
Vocabulary, Reading Comprehension, and Math Total; 

• path analyses, in which more complex causal models were developed and 
tested, in order co understand causal sequences, leading to Year 2 SAT 
outcomes ; 

• analyses of covariance, in which selected predictor variables were used 
as main effects in analyses of variance of Year 2 SAT scores, and other 
variables were used as covariates (in order to study interactions of key 
variables); and 

• focused subgroup analyses, in which the Year 2 SAT scores of subgroups of 
students, homogeneous on such variables as Raven (aptitude) and English 
SOPR (oral proficiency) were analyzed using simple comparison of means. 

For logistical and statistical reasons,' only a limited number of predictor 
variables could be used for these analyses (especially the path analyses, 
analyses of covariances, and focused subgroup analyses). In general, the 
variables used could be placed into five categories: 

(1) Student Characteristics 

(a) Raven Progressive Matrices (total adjusted score) 

(b) English SOPR total score 

(c) Indian SOPR total score 

(d) Year 1 SAT adjusted scores in Vocabulary, Reading Comprehension and 
Math Total 

(e) Pre-kindergarten school experience (no, yes) 

(2) Instructional Variables 

(a) Hours per week of instruction in English, special English, English 
reading, or mathematics 

(b) Percentage Indian language use in math, science, social studies, and 
ethnic heritage (or math only) 
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(3) School /Community Variables 

(a) Percentage of LEP students In grades K-6 

(b) Percentage of students who sometimes speak an Indian language In or 
around the school (categorized on a six-point scale) 

(c) Extent of Indian language use outside of classrooms by students, 
teachers, and principal (composite) 

(d) Indian language use In the surrounding community (composite) 

(4) Teacher Variables (M m average for main teachers in each of two years, A * 
average for all teachers in two years) 

(a) Use of Indian language outside of classroom (M,A) 

(b) Self-identification as Native American (M,A) 

(c) Years of teaching experience (M) 

(d) Whether teacher (as opposed to an aide) provides most of instruction 
in math, science, and social studies (M) 

(e) Proportion of instructors who are aides, not teachers (A) 

(5) Home /Family Variables 

(a) Parents* years of education (composite with more educated parent 
weighted three times as heavily) 

(b) Parents 1 use of English in the home (composite) 

(c) Presence of English language newspapers and magazines in the home 
(composite) 

(d) Hour 8 per week child spends: 

• being read to 

• reading 

• doing homework 

• watching/listening to TV/radio programs in English 

(e) Parent's ratings of Importance of school (categorized on a three point 
scale) 

(f) Parent's interest in child's education (composite) 

(g) Parent's expectations for how far the child will go In school 
(categorized on a five point scale) 

/ 
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4C C. SUMMARY OP ANALYTIC RESULTS 



There are a variety of potential explanations for the performance of Indian 
students, and data relevant to many, but not all, of them were collected as 
part of this study. Those for which data are available and analyses were 
performed can be thought of in terms of the following five categories: (1) the 
academic aptitude of the students; (2) the English language proficiency of the 
students; (3) the amount and language of instruction; (4) characteristics of 
the instructional staff; and (5) characteristics, and expectations of students 9 
families. Results from the analyses pertaining to these categories of 
potential explanation are summarized below, with more detailed descriptions of 
the analyses involved provided in Appendix C of this report. 

4C.1 ACADEMIC, APTITUDE 

One obvious possible explanation for low achievement test scores is the 
academic aptitude of the students. If the students 9 aptitude is low, one 
would er^ect low performance on achievement tests. However, as described in 
Cliapter 3, the maaa scores for students in this study Here not particularly 
low. Indeed, fct^yiwere equivalent to the scores of the publishers 9 large and 
presumably repr 3sentative TTaifced State* normative group. 

Mow specifically, two versions of the Raven Test of Progressive Matrices 
were used as the measure of academic aptitude in this study. The Raven was 
selected because it is not dependent oa language ability and it widely 
regarded as culturally appropriate for Ipdiari students (for example, see the 
discussion in Giliiland, 1986, p. 8). That there is an appropriately broad 
distribution of test scores across the study's students, and as would be 
expected, the Raven scores are positively correlated with the achievement test 
scores of the study 9 s Indian students (they are even a better predictor for 
students in Cohort D than Cohort A) provides further support for the 
appropriateness of the test. Thus, since the scores of students in Cohort A of 
this study were at the 58th percentile of the normative group and students. in 
Cohort B were at the 45th percentile, it is clear that academic aptitude does 
riot explain the low achievement test score?. 
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; 4C ♦ 2 ENGLISH LANGUAGE PROFICIENCY 

k second; possible explanation for low achievement test scores is a lack of 
English language skills among the Indian students. Low English language 
proficiency would be expected to impact especially on the Vocabulary and 
K> Reading Comprehension subtests of the SAT (but less so on Math subtests). 

Mean English SOPR scores were in fact considerably below native speaker levels 
(Cohort A - 19.5, Cohort B - 19.9 out of 25) and were correlated with SAT 
subtests in both years (see Table 4.1). 



TABLE 4.1. Correlations of English SOPR and SAT scores 







Cohort 


A 




Cohort B 






Reading 


Vocab. 


Math Total 


Reading 


Vocab. 


Math Total 


Year 1 


.492 


.609 


.530 


.567 


.380 


.449 


Year 2 


.486 


.459 


.451 


.450 


.395 


.408 


n 


- 182 


182 


"161 


140 


122 


113 



The correlations between English SOPR and SAT scores were also less for Tear 
2 than for Year 1 (except for Cohort B Vocabulary subtests). This is the 
pattern which would be expected If schools were teaching English skills in Tear 
2 which are relevant to the SAT tests, and thus the English SOPR was a less 
valid measure of English proficiency In Year 2 than in Tear 1. 



Low SAT scores can only partially be attributed to low English proficiency, 
however. Low English proficiency should have had more impact on Reading 
'Comprehension and Vocabulary scores than on Total Math scores, yet Indian 
students scored equally low on all tests (in comparison to national norms). 
Also, the mean English SOPR scores of Indian students were above those of 
LM-LEP students in the main LEP study (Toung et al. , 1986), yet students scored 
lower on Math SAT tests. 
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4C.3 INSTRUCTION 



A third possible explanation of low SAT scores concerns the amount or 
nature of instruction. Two types of variables were studied in this regard: 
(1) the amount (hours per week) of instruction in specific subjects; and (2) 
the extent to which the Indian language was used in instruction. 

Although there were a number of simple correlations between instructional 
variables and outcomes, these correlations were lowered to near zero once 
student background characteristics were controlled for. Thus, for example, 
students with low English SOPR scores received more special English 
instruction and scored lower on SAT tests, the negative relationship between 
special English instruction and SAT scores was artificially produced by a 
pedagogically sound approach of providing special English instruction to those 
most in need. 

The results of our causal analyses thus showed very few statistically 
significant relationships between instructional approaches and achievement 
test outcomes (see Appendix C). The instructional variables which were 
studied thus are probably not major causes of the Indian student 8 f low 
achievement scores. 

4C.4 INSTRUCTIONAL STAFF CHARACTERISTICS 

A fourth possible explanation of lo*r achievement test outcomes for Indian 

students concerns the characteristics of instructional staff. It is possible 

that f,he absence (or presence) of Indian staff, the experience of such staff, 
or the use of aides may be key factors in achievement outcomes. 

A number of such factors were studied in regression analyses and other 
special analyses. In general t instructional staff variables were not reliable 
predictors of outcomes (they had low simple correlations and highly unstable 
beta weights in regressions). Although there were a few interesting findings 
in special analyses (students in some subgroups appeared to do better if they 
had both Indian and non-Indian teachers) , the overall findings provide little 
explanation for low achievement test outcomes. 
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4C.5 HOME/FAMILY/ COMMUNITY CHARACTERISTICS 

A fifth possible explanation of low achievement test scores is the 
community and family environment in which Indian students live. Such factors 
as low English language use in the school and community, low parental 
education levels, and low educational expectations could all be factors in low 
Indian student achievement* 

A number of such factors were examined in the regression and path 
analyses* The results showed consistent and substantial correlations of : 
achievement test scores with six variables. Test scores were negatively 
correlated with: 

# community use of the Indian language ; 

# percentage of students speaking the Indian language; and 

# use of the Indian language outside of classrooms by principals, teachers, 
and students. 

Test scores were positively correlated with: 

•parents' use of English in the home; 

# hours per week the child spends reading; and 

# parents' expectations of how far the child will go in school. 

In regression analyses, the beta weights for these variables tended to be 
, unstable and less substantial. This was due to the fact that many of these 
predictor variables were lntercorrelated (especially those relating to Indian 
language use). Nonetheless, it appears that lack of exposure to English In 
the home, school, and community, the lack of reading In the home, and low 
parental expectations regarding education all may be factors In the low 
achievement test scores of Indian students. In this regard, It should be 
noted that parents of Indian students reported less reading by their children 
and had lower educational expectations than did parents of students In the 
main LEP study. Indian students had more exposure to English In the home and 
school, however, than did LM-LEP students in the main study (see Year 1 
Report). 
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r 4C.6 SUMMARY 

1 

The results of our analyses provide some clues concerning the low 
achievement test scores of Indian students In the study. Community use of 
Indian languages, and the subsequent low English proficiency of students, do 
put 3tudefits in Title VII Indian sites at an educational disadvantage* Lack 
of exposure to English is not a sufficient explanation, however. Indian 
students scored lower on Math subtests than did students in the main LEP 
study, even though their English oral proficiency ratings were higher. 

There Is also evidence that certain home/family characteristics may be 
important. Both the amount of reading at home b, the students, and parent s f 
expectations for educational level to be achieved, were correlated with 
achievement test outcomes, and were lower for Indian students than for 
students* in the main LEP study. Lack of home support for educational 
achievement may thus be an Important factor. 

On the other hand, the variables related to instruction and instructors 
which were selected for analysis do not appear to explain the low achievement 
outcomes. The fact that outcomes (In comparison to norms) were lower In the 
second year of the study than in the first suggest that there may be problems 
with the Instruction provided. However, identifying the nature of these 
problsss will eequire either further analysis of the existing data or new 
research on the Instruction of these students. Some thoughts on possible 
factors are presented in the next section of th1i chapter. 

In summary, two factors (lack of exposure to English and lack of home 
support for educational achievement) have been implicated in the low 
achievement test scores of Indian students in the study. Assessing the 
absolute and relative Importance of these and other factors (especially those 
relating to instruction) will require additional study, however. 
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4D D« IMPLICATIONS AND OTHER AVENUES TO EXPLORE 

4D.1 UNEXPLORED FACTORS 

In the preceding section we examined several variables which might have 
axplalned the poor academic achievement of sample studencs. As noted, only a 
partial explanation of why the achievement of these students was so low was 
found* Besides the factors discussed in the preceding section there are a 
number of others which conceivably were responsible for the poor performance 
of. the students, but which could not be included in the present study; they 
fall into two categories: (a) factors which are common to all sites avid thus 
could not vary, and (b) socioculturai factors on which it was not within the 
bounds of the study t\ collect extensive data. Each of these categories is 
discussed below* 



a* Factors Common to All Sites 

There were four variables which were common to all sites but which 
distinguish the sites from other school systems. These factors may explain 
low scores, but their Influence could only have been studied if comparison 
schools had been Included in this study. Thus, their Influence was impossible 
to determine. 



1) Socioeconomic status of the students 

This could not be studied because almost all of the sample students came 
from families with low to very low socioeconomic status. 

2) Integration versus segregation 

This could not be studied because the student populations at all of the 
schools included in the study are entirely or almost entirely Indian. 

3) Attendance at rural versus suburban versus urban schools 

The schools included in this study were all located in extremely remote, 
rural areas, thus precluding investigation of ;he effects of this 
variable on student achievement. 
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4) Residence on or near a reservation versus residence away from a 
reservation 

This important variable could not be studied because virtually all of 
the students included in this study had lived their entire lives on or 
near a reservation. 

On another note, the small size of the student sample also imposed 
restrictions on the search for explanations for the low achievement of the 
students. Several potentially interesting analyses of explanatory variables 
had to be abandoned when the number of available cases became too small. 



b. Unexplored Cultural and Sociological Factors 

Gilliland (1986, pp. 3-9) points to a number of socioculeural factors as 
potentially' responsible for the poor academic achievement of Indian students. 
These factors are: 

• Cultural differences (i.e., between the student f s native culture and the 
culture of the school); 

• Teacher's lack of understanding (i.e., lack of knowledge bj 
instructional staff of the student's native culture); 

• Differences in values (i.e., between the student and the teacher); 

• Differences in learning styles (i.e., the greater prevalence of 
"learning by model" versus "learning by rule" among Indian students, the 
reverse of what normally occurs in school); 

m Lack of motivation (i.e., lack of perceived relevance of knowledge 
obtained in school to life outside the school); 

• Differences in background and language (i.e., between the student and 
the teacher); 

• Home and community problems (i.e., problems which the student must deal 
with at home or in the community); and, 

• Use of inappropriate tests (i.e., use of culturally biased tests). 
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Sith regard to Gilliland' 8 final point > i.e., the "Use of inappropriate 
tests," two types of tests were administered to students in this study. They 
were: (1) a test of academic aptitude (the Raven Progressive Matrices) and (2) 
a test of academic achievement (selected subtests of the Stanford Achievement 
Test). The Raven Progressive Matrices is listed by Gilliland (1986, p. 8) as 
one of four examples of tests which are appropriate for use with Indian 
students, since it is a non-language test which requires no knowledge of any 
particular culture. 

The question of. the "appropriateness" of the subtests of the Stanford 
Achievement Test is somewhat more complicated to answer. All achievement 
tests measure cultural knowledge as well' as skills knowledge. The 
"Vocabulary" portions of achievement tests are explicitly and intentionally 
tests, of cultural knowledge since a word is not "known" unless its 
culturally-based meaning as well as its spelling and pronunciation are known. 
Similarly,' "Reading comprehension" subtests intentionally require a 
significant amount of cultural comprehension. Furthermore, while mathematics 
is an abstract, culture-free system, it is relevant in real-life situations 
only to the extent ihat it is associated wita culture bound behaviors such as 
buying/selling, banking, cooking, etc* Thus, mathematics tests also test 
cultural knowledge. It may be questioned whether Indian students living in 
remote rural areas on or near reservations to know general American culture as 
reflected in the SAT subtests; however, to the extent that it is a goal of the 
school to teach general American school subject knowledge — which includes 
teaching general American culture — the subtests of the Stanford Achievement 
Test are probably an appropriate measure of the acquisition of this 
knowledge. Furthermore, all the data we have indicates that teaching general 
American school subject knowledge is a goal of the schools Included in this 
study and, thus, that the SAT subtests are probably appropriate measures. 

Some data relevant to addressing the factors mentioned by Gilliland were 
gathered during this study* For example, data on student and teacher 
ethnicity were obtained and could be used to address the issue of "Teacher f 8 
lack of understanding," at least in part. (It would also be necessary to know 
the extent to which teachers nor of the same ethnicity as the student had 
acquired some understanding of the students 9 cul. are, a topic on which the 
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study did not obtain sufficient data*) Data were also collected on teacher 
language background, classroom organization, teacher attitudes towards the use 
of native language in instruction, use of special instructional materials, and 
other variables which may be relevant. Thus, further analysis of existing 
data would appear to be justified. 

Differences between the students 9 native culture and the culture of the 
schools or their teachers may explain, for example, the Indian students 9 poor 
performance in math. Davison and Schindler (1986) in their study of Crow 
Indian students conclude that three factors adversely influence the students 9 
ability to learn English language mathematics: (1) the role of language; (2) 
the students 9 culture; and (3) the students 9 learning style. With respect to 
culture and learning style, Davison .^nd Schindler conclude, 

"The influence of the students 9 culture, and the perceived relevance of the. 
mathematics curriculum, is seen as an additional problem. Except for working 
with money, students do not perceive the mathematics they learn in school to 
be of any use to them, nor is the school curriculum seen as culturally 
relevant. Host significantly, the students did not- share either a large 
number or a wide range of goals. The school curriculum, as far as these 
students were concerned, related to just one goal — earning money. Even 
though these students were youn^, school had very little meaning for them. 

"The methods by which mathematics is typically presented do not consider 
the Indian student 9 s learning style. Textbooks are typically written for 
white middle class America and present mathematics as an essentially abstract 
subject. While many textbook series now make reference to the use of tactile 
and visual aids, few teachers present mathematics in other than an abstract 
manner. The Indian student depends upon a more sensory approach to be able to 
learn mathematics effectively. " 

However, truly adequate investigation of the factors mentioned by Gilliland 
and those investigated by Davison and Shindler would require data collection 
methods which were outside the scope of the study. For example, several would 
be best addressed by the kind of in-depth, rich data obtainable only through 
such qualitative data collection methods as observations and ethnographic 
interviews. Such would be the case, for example, in addressing the topics of 
"Cultural differences," "Differences in learning styles," and "Home and 
community problems." Similarly, the investigation of other of these factors 
would require more extensive parent interviewing than was carried out as wexl 
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as interviews with study students, something which was not done in this 
study. This would be the case, for^ixampie, for "Differences in values," 
"Lack of motivation," and "Home and community problems." 

4D.2 NEED FOR FURTHER RESEARCH 

The most significant finding from the analysis of the data for 'the second 
year of this study is that the scores on academic achievement tests of 
elementary grade-level Indian, students attending school on or near Indian 
reservations are quite low, and that, these scores declined or remained the 
same over the two years of the study. This is true, even though the schools 
in the study were receiving and had received in the past federal funding 
targeted at improving student achievement. 1 

Although none of the schools in this study were producing students who were 
doing well in terms of national norms, there are some schools on or near 
Indian reservations where this is loc the case. There has been much effort 
and experimentation regarding the education of Indian children over the past 
20 years and reputedly there has been some success. Tt would be appropriate, 
therefore, to identify systematically those schools which are having 
verifiable success in terms of achievement test results and, in comparison to 
schools such as those in this study, to identify factors associated with that 
success. Based on the results of such a study, recommendations for 
improvements in the education of Indian children could be drawn. 



^All of the schools included in the study receive Title VII (Bilingual Education 
Act) funds. In addition, most receive ECIA Chapter 1, Title IV (Indian Education 
Act), Jobnson-O'Malley, or other federal funds intended to improve the academic 
achievement of disadvantaged students. 
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Appendix A: GLOSSARY 
Part 1. ABBREVIATIONS AND SPECIAL TERMS 



Term 

Academic instruction 



Adjusted score 



Algonquian-language students 



Cluster 



CPM 



Meaning 

Used in. discussion of instructional services 
to refer to math, science, social studies, 
and ethnic heritage instruction as distinct 
form instruction in language arts or other 
subjects • 

•j test score corrected for omitted items by 
adding to the number of items answered 
correctly a value equal to the quotient 
obtained when the number of items omitted is 
divided by the number of options per item. 

Students Whose native language is an 
Algonquian language (e.g., Atsina (Gros 
Ventre), Ojibwa, Passamaquoddy). 

A set of LEP instructional services received 
by a student at a given tine and defined in 
terms of the following five characteristics: 

.1) Percentage of use of the child's native 
language, in instruction in subjects 
other than language arts. 

(2) Whether special instruction in English 
is provided. 

(3) Whether simplified English is used more 
than regular English in instruction in 
math, science, social studies and 
ethnic heritage. 

(4) Whether simplified English is used more 
than regular English in teaching 
English language arts. 

(5) Whether instruction in native language 
arts is provided. 

There are 32 clusters. 

Raven Coloured progressive Matrices 
(Thi¥ was the level of the Raven 
Progressive Matrices Test used in grade 1.) 
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Term 



DK 



English-language students 

EP 

ESL 

Indian 



LEP 
IM 

IM-LEP 
IM-LEP Study 

Major cluster 

Navajo-language students 
Other Indian language students 

Raven 



Rights score 



SAT 



Meaning 

Don't Know 

~ (Response to questionnaire item) 

Students whose native language is English. 

Biglish-^roficient 

Englishes -a -Second Language 

Individuals (singularly or collectively), 
and their possessions, who are descended 
from one or another of the indigenous 
peoples of the Americas, exclusive of Aleuts 
and Eskimos • 

Limited-Siglish-jgroficient 
Language minority 

Language-minority liiirited-lfoglish-£roficient 

"National Longitudinal Evaluation of the 
Effectiveness of Services for Language- 
Minority Limited-English-Proficient Students* 

The six major categories in which the 32 
clusters are classified . 

Students whose native language is Navajo 

Students whose native language is an Indian 
language, but not Navajo or an Algonqulan or 
Slouan language* 

Raven Progressive Matrices Test 

Different levels were used In grades 1 and 
3—the CPM level in grade 1 and the SPM 
level in grade 3. 

A test score equal to the number of Items 
answered correctly* 

Stanford Achievement Test 

.Standard deviation 
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Bart 2. STATISTICAL NOTATION 



M 
N 
n 
c 

S.D. 
r jk 

Hi: 
8 



R* 

0 

A. 

I 

p 

R* 
R 1 



Frequency 
Mean 

Number of cases 

Number of items in test 

Number of choices per multiple-choice item 

Standard deviation 

Correlation between variables j and k 

(Unless otherwise specified it is the Pearson product- 
moment coefficient .) 

Reliability of variable i 

Mean of variable X 

Standard deviation of sample 

(This is the standard deviation obtained using N as 
the divisor.) 

Value of s for variable X 

Estimate of populatxon standard deviation 

(This is the standard deviation obtained using N-l, 
or number of degrees of freedom, as the di^dsor.) 

Value of CT for variable X 

*Rlghts score (i.e., number of test items answered 
correctly) 

Number of test items omitted 

Number of test Items attempted 

Adjusted score (i.e., score adjusted for omitted items) 
Standard multiple regression weight (beta weight) 
*Multiple correlation coefficient 

Multiple correlation coefficient adjusted downward by Wherry 
skrinkage formula 



Note the two definitions of R. Context makes it clear which one is appropriate. 
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Bart 2. STATISTICAL NOTATION 



M 
N 
n 
c 

S.D. 
r jk 

Hi: 
8 



R* 

0 

A. 

I 

p 

R* 
R 1 



Frequency 
Mean 

Number of cases 

Number of items in test 

Number of choices per multiple-choice item 

Standard deviation 

Correlation between variables j and k 

(Unless otherwise specified it is the Pearson product- 
moment coefficient .) 

Reliability of variable i 

Mean of variable X 

Standard deviation of sample 

(This is the standard deviation obtained using N as 
the divisor.) 

Value of s for variable X 

Estimate of populatxon standard deviation 

(This is the standard deviation obtained using N-l, 
or number of degrees of freedom, as the di^dsor.) 

Value of CT for variable X 

*Rlghts score (i.e., number of test items answered 
correctly) 

Number of test items omitted 

Number of test Items attempted 

Adjusted score (i.e., score adjusted for omitted items) 
Standard multiple regression weight (beta weight) 
*Multiple correlation coefficient 

Multiple correlation coefficient adjusted downward by Wherry 
skrinkage formula 



Note the two definitions of R. Context makes it clear which one is appropriate. 
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Appendix B. STUDY DESIGN AND INSTRUMENTATION 1 
1. INTRODUCTION 



The design for this study called for a two-year longitudinal evaluation, 
modeled after the study design of the H National Longitudinal Evaluation of the 
Effectiveness of Services for Language-Minority Limited-English-Proficient 
Students, H The first part of the study was focused on describing the services* 
offered to American Indian limited-English-proficient (LEP) students in 
elementary schools which receive Title VII funding* The: second part of the 
study looked at the academic performance of American Indian LEP students. 
Title VII projects were selected as the study focus because they would provide 
an accessible source 01 Indian LEP students and because there was interest 
within the Department of Education in a description of Title VII project 
services for Indian students. The conceptual framework, sampling plan and 
instrumentation for the first year of the study were described in Appendix C 
of Rudes et al. (1988), Here we describe the sampling plan and 
instrumentation for the second year of the study* 

B2 2. STUDY DESIGN 

B2.a THE SAMPLE 

The basic research plan for this study called for two cohorts of students 
in a national sample of schools served by Title Vll-funded projects. The 
first cohort (Cohort A) consists of students who were in grade 1 during the 
1985-86 school year. The second cohort (Cohort B) consists of students who 
were in grade 3 that year. 



Abbreviations and other special terms used in this study are defined in the 
glossary in Appendix A. 
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The sample for the main on-site data collection during the first year of 
the study consisted of 23 of the 56 funded Title VII projects serving Native 
American students in the elementary grades during the 1986-87 school year. 1 
These 23 projects served a total of 1588 first and third grade Indian students 
who came from 16 different tribal groups, and who had 18 different native 
language backgrounds* For the sample for the second year of the study 8 of 
these 23 projects were selected. The specific projects which participated in 
the second year of the study are shown in Table B.l. These 8 projects served 
278 Cohort A and 210 Cohort B students who, as shown in Table B,2, came from 
7 different tribal background and who, as shewn in Table B.3, had 7 different 
native langauges. 

B2.b DATA COLLECTION INSTRUMENTS AND INFORMATION SOURCES 

At the 23 sites visited during the first year, data were collected 
regarding school districts, schools, principals, instructional personnel, 
parents, community leaders, and students* During the second year of the 
study, data were gathered only from sample students and their teachers* 

Much of the data collection during the first year focused on "control 
variables *\ The need for control variables in such a study is critical. The 
term "control variable" as used here refers to a variable that; helps prevent 
distortion of the results that might otherwise occur from different 
instructional programs as a consequence of different levels of ability and 
potential among the students in the groups being compared, or other factors 
extraneous to the focus of the study. 



3-Note that, because of the high cost of data collection in Alaska, the decision 
was made by the U.S. Department of Education to exclude projects in Alaska from 
the main data collection for this study. In addition to the 23 projects included 
in the main data collection £6r the first year, data were gathered using case 
•study methodology on two California projects. The findings on these two projects 
were reported separately, in Appendix A of Rudes v, al . (1988). 
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TABLE B.l; The eight Title VII projects participating in the 
on-site data collection for Tear 2 



White Mountain Apache Tribe (White Mountain Apache Reservation , Arizona) 

Peach Springs School (Hualapai Reservation, Arizona) 

Kaibeto Boarding School (Navajo Reservation, Arizona) 

Central Consolidated School District #22 (Shiprock, New Mexico) 

Dunselth School District #1 (Dunoeith, North Dakota) 

Loneman School Corporation (Oglala, South Dakota) 

School District 17H (Hardin, Montana) 

Mississippi Band of Choctaw Indians (Philadelphia, Mississippi) 



TABLE B.2. Number of students in the sample by 
tribal group 


Tribal group 


Number of Students 


1. Navajo 


208 


2. Choctaw 


60 


3. Mitchif (Metis) 


59 


4. Crow 


48 ' 


5. Hualapai 


47 


6. Oglala (Lakota) 


34 


7. Apache 


32 


TOTAL 


498 



TABLE B.3. Number of students in the sample by 
native language 



Language Number of Students 

1. Navajo 174 

2. English 99 
3* Choctaw 60 

4. Crow 47 

5. Hualapai 42 
6* Lakota 33 

7. Apache 32 

8. Ojlbwa (Chippewa) 1 
TOTAL 498 
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Various different kinds of contr< 1 variables were deemed desirable. These 
included a baseline measure of academic ability level independent of the 
child's language, an evaluation of the child's degree of oral proficiency in 
English and in the Indian language and measures of achievement in English and 
mathematics. Also included are measures of home context which prior research 
suggests may confound the effect of the instructional treatment variables of 
primary interest. The first of these variables (the baseline measure of 
academic ability) is provided by the Raven Progressive Matrices, the second by 
the Student Oral Proficiency Rating (SOPR), and the third by scores on the 
Bullish and mathematics subtests of standardized achievement tests which the 
students took last spring. The home context measures are provided by a 
questionnaire developed specifically for this study. Detailed descriptions of 
these instruments appear in Appendix C of Rudes et al. (1988). 

Other instruments used during the first year were for the purpose of 
describing the instructional treatments received by each student, the 
characteristics of the providers of those treatments, or their educational 
context. Each of these measures was either developed for the "National 
Longitudinal Evaluation of the Effectiveness of Services for Language- 
Minority Limited-English-Proficient Students," and modified as appropriate for 
this study of Native American students * or developed specifically for this 
study. Again, details of these instruments are provided in Appendix C of 
Rudes et al. (1988). The specific instruments which were used for data 
collection during the first year of the longitudinal study are shown in 
Exhibit B.l. 

During the second year of the study, the Student /Teacher Data Form and the 
Student Instructional Language Record Form were again completed by teachers of 
sample students to provide the study with current information on the 
instructional services being received by the students. In addition* the 
appropriate levels of the Stanford Achievement Test math and English subtests 
were administered to study students. 
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EXHIBIT B.l. Study instruments 



Instrument 
Project Director Questionnaire: 
School Statistical Summary Form: 

School Principal Questionnaire: 

School Policies and Procedures Form: 
Instructional Staff Questionnaire: 

Support Staff Questionnaire: 

Student/Teacher Data Form: 

Student Instructional Language Record: 

Student Performance Record: 

Parent/Home Questionnaire: 

Home /Community Language Use Form: 

Student Background Questionnaire: 



Completed by 

Title VII project directors 

Development Associates staff 
from school records and 
reviewed by school principals 

the principal of each school 
participating in the study - 

Development Associates staff 

all teachers of content 
subjects who work with students 
in the acudy sample 

all aides 9 tutors, volunteers, 
or resource staff who work with 
students in the study sample 

the homeroom or main teachers 
of each of the students in the 
study sample 

all teachers of content 
subjects who work with students 
in the study sample 

the homeroom or main teachers 
of each of the students in the 
study sample 

one parent or guardian for each 
student in the study sample 

a sample of parents of study 
students, and a sample of 
tribal leaders at each site 

Development Associates staff 
members from student records 



Stttdent Oral Proficiency Rating Form 
(English) : 



the homeroom or main teacher, 
or another teacher or aide who 
is fluent in English 
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EXHIBIT B.l. Study instruments 


9 continued 


Instrument 


Completed by 


Student Oral Proficiency Rating Form 
(Native American Language): 


the homeroom or mair* teacher, 
or another teacher or aide who 
is fluent in the language on 
which the child Is being rated 


Academic Aptitude Measure (Raven 
Progressive Matrices) 

Coloured Progressive Matrices: 


each first grade student in the 
sample 


Standard Progressive Matrices: 


each third grade student in the 
sample 


Stanford Achievement Subtests 
Vocabulary 

Reading Comprehension 
concepts or numoer 
Math Computation 
Math Applications: 


all of the students in the 
study sample. 
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3. COMPOSITE VARIABLES 



Before the data analysis was begun, a number of composite variables were 

developed, in most cases by combining on an a priori tiuttis selected 

questionnaire items dealing with the same topic* 1 Formation of many of the 
2 

composites began at the time the questionnaires and rating scales were 
being developed. Using a composite of several .questionnaire items dealing 
with the same general area, rather than using the individual items themselves, 
has at least two advantages* Fire", the composite (a weighted or unweighted 
sum of several items) is likely to be more reliable than any of the individual 
items; and second, using a composite often makes the findings more 
comprehensible and easier to interpret* 

When a composite is to be developed, it is necessary to decide whether it 
should be done on an> a priori basis or empirically* A wide variety of 
statistical methodologies exist for developing, composites empirically (e.g., 
multiple regression, multiple discriminant analysis, factor analysis), but 
in a study such as the present one there are sound arguments against each of 
them* A priori composites have the advantages of greater comprehensibility, 
convenience, and credibility, and they have an additional advantage in that 
they make better use of available data, since they do not require a set-as~de 
subsample* Thus, this approach, rather than a more empirically driven one, 
was adopted for developing most of the composites presented in this report* 



a few casl the composite was formed by combining ratings on rating scales 
or scores on tests, rather than responses to questionnaire items* 

^These composites are generally described briefly at the point in the report 
where their use in data analysis is reported* Some are described in somewhat 
more detail in Appendix D of Rutes et al. (1988). 
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B4 4. SCORING OF TESTS 

: o 

Because the present study is self-contained) incorporating its own control 
variables, it is noj .dependent on published norms in order to evaluate 
results. This gives the liberty to modify the scoring procedures used by 
the test publishers in standardizing their tests where we have reason to 
believe that the modifications may increase the validity and usefulness of the 
results • We have taken advantage of this circumstance to make some minor, but 
we think useful, changes . % It should be noted that implementing these changes 
will not impair the results in any way, since in addition to obtaining scores 
by the modified procedures we have also obtained the conventional set of 
rights scores. These latter will serve a useftil purpose, in that they will 
make it possible to use publishers 9 norms. 

B4.a KINDS OF SCORES 



s 



Both the Stanford Achievement Test (SAT) and the Raven Progressive Matrices 
are normally given scores equal to the number of items answered correctly 
(hereafter referred to as "rights scores w ) ; among items not answered 
correctly, no distinction is made between omitted items and items answered 
incorrectly. This mode of scoring a multiple-choice test assumes that every 
student answers every item. When that assumption does not hold, the child who 
omits items if he or she is uncertain of the answer is penalized inequitably; 
the child who makes a guess on all such items will probaiily get about a third 
of them right purely by chance if they are three-choice items, a fourth if 
they are four-choice items, etc., while the child who omits deprives himself 
of this advantage. One way of handling this problem is to "correct n the 
rights scores for omitted items by adding to the score the estimated number of 
items the child would have gotten right by chance had he made a guess rather 
than omitting the items. We choose to call the score obtained this way the 
"adjusted score." 



In our judgment, using adjusted scores is superior to using rights scores. 
To express this judgment in somewhat more technJ^l terms, adjusted scores 
tend to give a more valid indication of the student's level of knowledge or 
ability than do rights scores t If none of the examinees omits auy items, it 
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makes no difference which mode of scoring is used, because the rights score 
and the adjusted score are exactly equal; but to the extent that children 
differ in their tendency to omit items when they do not know the answer, it 
can make a big difference. Because using adjusted scores instead of rights 
scores has no effect (and therefore can have no ill effect) when no items have 
been omitted, and because it can represent, a major improvement — an increase 
in fairness — when items have been omitted by some children while Dther 
children have answered every item, whether they know the answer or not, we 
decided: Ko use adjusted scores as the principal scores for both the Stanford 
Achievement Tests and the Raven. However, as indicated above, we decided to 
also make a record of the rights scores, to permit comparison with the norms 
developed by the author or publisher. 

As has been implied, rights scores have been used as the basis for norms 
and other statistics provided by the test publishers or authors. Those who 
prefer rights scores base their preference on the belief that in scoring tests 
by hand It is easier to obtain rights, scores than adjusted scores, and that on 
theoretical grounds It does not make much difference «hlch kind of score is 
used since the correlation between theia is typically very high. However, in 
the present case all scoring Is done by computer, and even when the 
correlation between rights and adjusted scores are very high, there are still 
likely to be some children who omit large numbers of Items, which can 
substantially distort the results not only for the children affected but for 
research analyses that include these scores. Thus In subsequent chapters when 
we report data involving test results, those data, except where Indicated to 
the contrary, will be adjusted score data. 

B4.b SETS OF VARIABLES SCORED 

There is a slight difference between the list of tests from the Primary 1 
SAT battery (used ir. grad£ 1) and the Primary 3 battery (used in grade 3). In 
the latter the following tests are used: 
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vocabulary 

Reading comprehension 
Concept of number 
Math computation 
Math applications 



In the Primary 1 battery, on th< other hand, the last two of these five 
areas are combined in a single test, "Mathematics Computation and 
Applications • " To facilitate comparison of grade 1 and grade 3 results, we 
have scored the 22 Primary 1 computation and the 2* applications items 
separately as veil as together; and in the Primary 3 battery, we have 
obtained a combined score for these two tests is well as scoring them 
separately* 

For somewhat similar reasons we have also slightly expanded _ae set of 
scores, obtained for the Raven* The Raven Standard Progressive Matrices (SPM) , 
which is given in grade 3, consists of five sets of 12 items each — Sets A, 
B, C, D, E — Set A being the easiest and Set E the t'ost difficult* The 
Coloured' Progressive Matrices (CPM)* given in grade i, consists of three sets 
of 12 Items each — Sets A, AB, and B. Se A and B are identical to the 
like-named sets in the SPM except that in the CPM the Items axo colored* 
Since the sole function of the coloring is to serve as an attention-grabber 
for the very small children for whom the CPM is Intended, and since the colors 
provide no clue to the answers, we obtained separate scores for A+B in both 
CPM and the SPM* The purpose is to facilitate direct comparison between 
grades 1 and 3 on an identical set of Raven items* 

Table B. 4 summarizes the scores obtained and other miscellaneous 
information about the Raven and SAT tests* 
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TABLE B«4« Miscellaneous information about Raven Progressive Matrices 


and Stanford Achievement Tests 






Kinds 


1 of 


No. of 






Score 


Options 






Obtained* 


Per Item 


Number of Items 


Raver Progressive Matrices 










Coloured (CPM) 










Sets A + B 


A R 


I 


6 


24 


Sets AB 


A R 


I 


6 


12 


Total (A + AB + B) 


R 


I 


• 


36 


Standard (SPM) 










Sets A + B 


A R 


I 


6 


24 


Sets C + D + E 


A R 


I 


8 


36 


Total (A + 3 + C + D + E) 


R 


I 




60 








Primary 


Primary Primary 








Level -* 1 3 


1 3 


Stanford Achievement Test 








• 


English 










Vocabulary 


A R 


I 


3 4 


JO Jo 


Reading Comprehension 


A R 


I 


3 4 


40 60 


Total 




I 




78 98 


Hath 










I Concepts of Number 


A R 


I 


4 4 


34 34 


Computation 


A R 


I 


4 5 


22 42 


Applications 


A R I 


4 5 


23 38 


Computation + Applications 


R 


I 




45 80 


Total 


R 


I 




79 114 


Total (English + Math) 




I 




157 212 


*Code for "kind of score" 










A ■ No. of Items attempted 








R ■ No. of Items rl«ht 










1 ■ adjusted score 
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Appendix C. THE SEARCH FOR CAUSALITY: METHODOLOGY AND DATA 



.1 1. INTRODUCTION 

This appendix focuses on efforts to determine the extent to which various 
factors are related to achievement In three areas (English language 
vocabulary, reading In English, and mathematics) and to draw Inferences as to 
which of these relationships,, if any, may be "causative." We have put the 
word "causative" in quotation marks because, as is agreed by many / 
Statisticians, the question of what causes what is such a complex one that In 
most circumstances causation is virtually impossible to determine 
definitively* Nevertheless we have brought to bear a broad array of 
statisticians* tools — crosstabulations, breakdowns of aeans and standard 
deviations by subgroup, Correlations, multiple regression, analysis of 
variance, analysis of co variance, and path analysis - in an effort to at least 
throw some light on the question, with respect to Indian schoolchildren* The 
factors Investigated, as probably (or at least possibly) having some 
relationship to student achievement, fall In the following seven categories: 

1. Initial status of students ("control variables") 
2* Instructional variables 
3. Previous scores on achievement tests 

Note that this category logically belongs with category 1; however it 
was found more convenient to treat It separately* 
4* School variables 
5* Teacher variables 
6* Home-and-family variables 
7. Preschool 

The variables In each category are described in Chapter 4 Section B, 
together with, a similarly brief description about each of the achievement 
measures* 
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C.2 



2. ANALYSIS 0? THE DATA 



C.2a kiGRESSION ANALYSIS 

As a start in investigating the role of the student Y s Initial status and 
the Instruction received, correlation matrices were run and used as the basis 
of multiple regression analyses against the sthree criterion variables 
Vocabulary, Reading Comprehension, and Math Total. This was done separately 
f or iSfv *ets of independent variables - "Set I," consisting of the first three 
of the seven categories above, and "Set II," consisting of categories 4-7. 
For the Set I variables* 12 separate multiple regression analyses were run (3 
criterion variables x 2 cohorts x 2 years). For Set II it was only necessary 
to run 6 analyses; this was because the Independent variables In this set 
(essentially school, teacher, and family variables) either were stable enough 
that data on them only had to be collected once or else, if they were likely 
to differ from year to year, they were averaged over the two years. Therefore 
the year 2 criterion measures were the only ones needed for this set of 
multiple regressions. 

Table C.la shows the Set I regression analysis results for*two independent 
variables - SAT Vocabulary and SAT Reading Comprehension - and Table C.lb 
shows the corresponding results for SAT Math. As shown in the two tables, the 
12 multiple correlation coefficients rang-? from a low of .50 to a high of 
•83. When adjusted downward to make the multiple correlations comparable 
to each other even when based on different numbers of cases and different 
numbars of Independent variables, the reductions turn out to be quite small 
(.01 to .04) and the corrected values still quite large (.46 to. 81). 
Inspection of the two tables reveals that the largest Year 1 standardized 
regression weight (t>2ta weight) in almost every analyses Is for English 30PR. 
For the Math criterion, the Year 1 Raven betas are also quite substantial, 
particularly for grade 3. In most cases ti»e Indian SOPR has a negative 



Adjusted by the Wherry shrinkage formula, which corrects for the effect of 
number of independent: variables and number of cases. (The resulting reduction in 
the multiple R 1 8 greatest when the number of cases is small and the number of 
variables large.) 
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regression weight (as well invariably having a negative correlation with the 
criterion variable). The chief erseptions are that the regression weight is 
+•13 for grade 2 Math, and +.15 for grade 1 Reading Comprehension* 

A large part of the difference litween Year 1 and Year 2 results is 
undoubtedly due to the inclusion of the SAT Year 1 scores as independent 
variables in the Year 2 analyses* In these analyses the SAT variable 
corresponding to the criterion variable has a high beta weight in each 
^ regression analysis* 

Between them, the "control variables' 9 and the SAT Year 1 scores reera to 
account for almost all of the criterion variance that can be predicted by Set 
I variables* That leaves little or nothing for the instructional variables* 
Thus both the amount of instruction provided and how that instruction is 
divided between presentation in English and in an Indian language tend, in 
most cases, to have only a negligible beta weight* 

Using as the basis adjusted multiple correlation coefficients obtained as 
successive variables or subsets of variables are added, the corresponding 
incremental change in percentage of criterion variance accounted for has been 
computed* These percentages are shown in Table C*2, in the columns headed by 
H A W « Each successive value of represents the percentage of total 
criterion variance accounted for by the indicated independent variable or 
subset of variables after all the preceding variables have been taken into 
account* Thus, for instance, in grade 3 that part of Indian SOFR titat is 
independent of Raven and English SOPR accounts for 4*5 percent of the 
Vocabulary variance* This table shows clearly the preponderant effect of 
English SOPR, and in the case of Year 2 data the Year 1 SAT scores, on the 
Vocabulary and Reading Comprehension criteria* Raven also k*s a substantial 
effect in some cases, particularly in Cohort B* As for the Math criterion, 
the Raven plays a much more important role than *or either ,o£ the other two 
criteria; in the case of the Cohort B data it accounts for over one-fourth of 
the Math variance* 



ERLC 



DEVELOPMENT ASSOCIATES* INC. 



4 



Tab') Xi2 Adjusted Multiple correlation coefficients and percentage of dependent variable variance explained by the*, when various 
aubsets of independent variables are Included) 

MOTE* This table is baaed on the sase data as Tables C.la and C.1&. 

HOTATIOM , 

1 " — ™ * 

ri » number of independent variables on which R* la based. 

R 1 - adjusted Multiple correlation cccff Sclent (adjusted using Wherry shrinkage formula; aee footnote in Table C.la). 

100 (R 1 ) m percentage of variance accounted for by R 1 . 
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The results of the six multiple regression analyses involving the Set II 
independent variables (home, school, teacher, etc*) are shown in Table C.3. 
It must be recognized of course, that there is undoubtedly sose correlational 
overlap between the two sets of variables. Each of the six analyses had 21 
independent variables: 3 school variable?, 7 teacher variables, one preschool 
experience variable, and 10 home-and-family variables* As shown in Table C«3, 
the adjusted mtitiple correlation coefficients for these data are much; smaller 
than* the raw values. This is to be expected when, as in the present 
situation, the number of cases is relatively small, and, the number of 
independent variables large in relation to the number of cases. There is no 
analogous correction formula available for the regression weights, although 
they, too, are affected; therefore they should not be taken at face value. 

As seen in Table C.3, only about six of the variables have fairly 
substantial correlations with the criteria. For three of the six variables, 
the correlations are negative; for the remaining three, positive - as follows: 

• Variables negatively correlated with criteria: 

- Percentage of stud' its speaking an Indian language (Row 2 of table) 

- Extent o: tndian language use (by students, teachers, and 
principal/, in and around the school (Row 3) 

- Whether the classroom teacher (as opposed to aides) provides all 
the non-language-arts instruction 

• Variables positively correlated with criteria: 

- Parents 9 use of English in the home (Row 13) 

- Hotr.s per week the child spends reading, apart from homework 
assignments (Row 16) 

- Parental expectations regarding how far the child will go in school 
(Row 21) 

Most of these variables, however, do not have substantial beta weights - or 
if they do, the beta weights are not all in the same direction. The principal 
exception is for the Row 2 variable (percentage of students speaking an Indian 
language), for which most of the beta weights are substantial, and all six are 



Table C. 3 Multiple regression analyses for predicting , Year 2 SAX scores 
frost various school, teacher, and farily variables 

Mote: Each of the 8 suit' pic regression analyses is based on llstvlse data. 
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no. of cases 

average of main teachers 1 responses 
aggregation of all teachers - main and othar 
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negative. The Row 1 variable (percentage of LEP students in grades K-6) has 
by far the numerically highest beta weight in most of the six regression 
analyses , but one of these high weights is negative (and one weight is 
essentially zero). Other oddities are that the Row 4 and Rew 5 variables, 
which are essentially th? same variable except that one is for the main 
teacher and the other is for aggregated teachers, tend have betas with 
opposite signs 'and in some analyses these contradictory betas are numerically 
very lirge). Tha same phenomenon also occurs for another pair of variables 
(the Row 6 Row 7 pair). All these peculiarities are probably explained by the 
"bouncing beta" 9 phenomenon - the tendency for beta coefficients to be very 
unstable when intercorrelations among the affected variables are high, 
particularly when the number of cases not large. In view of all this 
instability we recommend not paying too much attention to the beta weights. 
(The correlations coefficients are much more stable and in a certain sense 
more informative; \ jce the beta ^eights they are shown in Table C.3.) 

Nevertheless, in spite of the questionable nature of some of the beta 
weights, it seemed advisable to check fur^ er on two of the variables for 
which these weights were substantial - the Row 4 variable (Teacher 
Questionnaire item 28a) and the Row 6 variable (Teacher Questionnaire item 
10a) - to determine whether analyses of variance wit*i two independent 
variables (the questionnaire item response of the child' s teact er and the 
child's Raven score) would reveal existence of a rt^I relationship (e.g., a 
significant effect for the questionnaire item). Table C«C*. shows the results 
for question 10a and Table C.4b ^hows corresponding data for question 28a. 
(The exact wording of the questions is shown at the top of the tables.) The 
main effect for question 10a was significant for two of tne six analyses of 
variance (two cohorts, three criterion variables); for question 28a none of 
the six was significant. 

Because of the mildly encouraging results for question 10a we went one step 
further for the two sets of data for which the main effect was significant, by 
obtaining means aud standard deviations on the dependent variable for each of 



"The #05 level of significance was used as the cutting point. 
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TABU C.4a Twoo a y oaalyalo of virtue* of stodaato' yaar 2 scoras oa SAT Vocabulary 
Baa dla* Coaarohoaaloa , aad M»th Total, with tsvaa total aad rupeuM 
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three mean response levels on the questionnaire item. The results (shown in 
Table C.5) indicate that the relationship i3 not unidirectional* The means 
are markedly higher for children who had had at least one Indian teacher and 
one non-Indian teacher than for children all of whose teachers, or none of 
whose teachers, were Indian* The explanation for these findings is not 
evident. But it may be relevant that the significant effects occur only for 
Reading Comprehension and Mathematics - two subjects in which the teacher 
plays a major role in transmitting skills - as opposed to the kind of 
vocabulary measured by the SAT Vocabulary test, which in normal circumstances 
is learned primarily outside of school* 

C.2b PATH ANALYSIS 

Path analysis, which is. an elaboration of multiple regression, was the next, 
step in the analyses* Separate path lodels were developed for Year 2 adjusted 
scores on the Vocabulary, Reading Comprehension, and Mathematics Total 
portions of the Stanford Achievement Test* Theoretical models were developed 
and wore tested using procedures suggested in a paper by the United States 
General Accounting. Office (1982).* This approach uses regression results from * 
all intermediate variables and from tHe final outcome variable, and uses 
standardized regression coefficients for unidirectional path coefficients and 
simple bivariate correlations for bidirectional path coefficients* 



The variables used in the path models were: 



• Community Language Use; 

• Raven: adjusted total score; 

• Hours English Instruction (Year 1) ; 

• Percentage Indiarv Language Use in Math, Science, Social Studies, and 
Ethnic Heritage [or Math only] (Year 1); 

n SAT [Vocabulary, Reading Comprehension, o;: Math Total] (Year 1): 
relevant adjusted score on the Standard; Achievement Test in Year 1; 



*U.S. General Accounting Office, Institute for Program Evaluation. "Methodology 
Transfer Paper 1. Causal Analysis: A Method to Identify and Test Cause and 
Effect Relationships :in Program Evaluation^ q Washington, D.C. : February 1982\ 
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•> SOPR English: total score; 

• SOPR Indian Language: total 'score; 
t Hours English Instruction (Year 2); 

• Percentage Indian Language Use in Math* Science, Social Studies, and 
Ethnic Heritage [or Math only] (Year 2); and 

• SAT [Vocabulary, Reading Comprehension, Math Total] (Year 2): relevant 
adjusted scores on the Stanford Achievement Test in Year 2. 

Theae variables are described in Chapter A, Section B. 

The results of the. path model analyses are 3hown in Figures C.l to C.6. 
The path analyse? do not greatly expand the insights from the multiple 
regression analyses described in the preceding section. The path analyses do 
suggest that: 

• treatment variables (hours of Instruction, percentage Indian language 
use) are relatively^waak preductors of Stanford Achievement ies* . Homes; 

• Raven scores are somewhat stronger predictors for the third-grade cohort 
than the first-grade cohort; 

• the strongest predictors of SAT test scores are other test scores or 
ratings (previous SAT tests, Raven tests, oral proficiency ratings) • 

In addition to the above findings concerning the uirect prediction of 
achievement test outcomes, there are two findings which are related to 
non-criterion variables : 

• community language use is more strongly correlated with Indian language 
oral proficiency than with English oral proficiency (indicating that 
Indian language proficiency is less variable within sites than is English 
proficiency); and 

• treatment variables are moderately predicted by community languages use. 
C*2c COVARIANCE ANALYSIS 

Since analysis of variance (or covariance) may show significant effects 
where multiple regression and its variant, path analysis, do not, the next 
step was to run analyses of covariance, in an effort to determine whether the 
language of instruction really had as little effect, as the multiple regression 
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results made it; appear to have. In preparation for these analyses, Tables 
C.6a, C.6b, and C.6c were provided, shoving the means and standard deviations 
of SAT Vocabulary, Reading Comprehension, and Math respectively, for subgroups 
fairly homogeneous with respect to English SOPR (divided in. 3 categories) , 
Indian SOPR (4 categories) , and percentage of time English is used in 
instruction (2 categories). The percentage-of -time variable, as in the case 
of the multiple regression analyses, is for ma^h instruction when the 
criterion variable is SAT Math, and for math,, science, social studies, and 
ethnic heritage instruct* >n combined when the criterion variable is SAT 
Vocabulary or Reading Comprehension. Because for most students, the Indian 
language was used relatively little, if at all, percentage of English use was 
dichotomized at 89% (a rather high point on the O-to-100 scale). 

Separately *or each cohort for each year of the study (i.e., separately for 
each grade) an analysis of co^riance of the percentage-of-time-in-English 
dichotomized variable was carried out for each of the subgroups homogeneous 
with respect to the English and ludian SOPRs. The two covariates used in the 
Tear 1 analyses were (1) Raven total and (2) hours per week of instruction in 
English (in the case of the Vocabulary aad Reeding Comprehension criteria) or 
in math in the case of the Math criterion. For the Tear 2 analyses there was 
also a third covariate, the year 1 SAT score corresponding to the criterion 
variable. The results of these 12 covariance analyses (3 criterion variables, 
4 grades each) are summarized in Table C.7. In this table the F ratio, 
degrees of freedom (df) ? and significance level (p) are shown for each 
covariate, eatf main effect, and all interactions. The results for the most 
part confirm the multiple regression findings. The Raven is consistently 
significant (at the 99 .00" level, i.e., under .005) for all Tear 1 analyses; in 
Cohort B it is also significant (at the .05 level or less) in the Tear 2 
analyses. It is perhaps noteworthy that number of hours per week of English 
instruction is sporadically significant £t the .05 level (twice out of eight 
analyses) as is the corresponding value for mnt^ instruction (one out *>£ four 
analyses), in spite of the generally negligible magnitude of the corresponding 
beta weights in Tables C.la and C.lb. Like the Tear 1 F values frr English 
SOPR, the F. values of SAT covariates in the Tear 2 analyses arc all 
significant at the ".00" level. 
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31.71 
12.10 
21 


32.50 30.57 24.25 
6.25 12.09 9.95 
4 7 4 


30.75 
U.24 
36 


30.66 
10.80 
74 


24.93 
7.52 
15 


23.00 
5.48 
5 


28.00 29.00 20.40 

4.62 

115 


22.83 
5.22 
12 


24.00 
6.56 
27 

31.92 
U.19 
142 
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TABU C.te Mum and otaadard daylattoaa of S.A.T. Hath Total 

for stttdtats claaaiflad on tha bull of logilth SOU, Indian SOHt, 
tod oatcantat* «M of EnfUth Itnrufo In taaching aath 




4 1 



k 2 



13-17 18-22 23-25 



Matty 1 23-25 H 52.25 55.57 72.00 62.00 

form t o* 9.71 7^93 ~ — 

n 4 7ii 

18-22 M 47.11 44.70 48.15 52.20 

o- 13.41 8.01 11.66 8.23 

K 19 23 20 5 

5-17 M 35.61 38.75 40.57 44.56 

<r 9.33 10.24 13.07 12.38 

n 9 4 7 9 



Total. M 
o* 
* 

Priaary 2 23-25 M 
Pots 8 a* 

If 

18-22 H 

<r 

H 

5-17 H 

n 

' Total M 
o* 
H 



X of tla* Ea*liafc ii oaad la inatractioa — r— 

89. OX or laaa Worm than 89.0X 

OTaTUJH ~~" 



Primary 3 
Fots F 



23-25 



18-22 



5-17 



47.00 57.50 61.00 61.00 
*— 31.82 — 2.83 

1. 2 1 2 

84.00 63.50 53.14 

12.02 16,31 

12 7 0 

49.25 37.00 41.71 53.75 

6.34 12.65 16.29 

4 17 8 



63.33 69.00 74.00 69.86 
10.41 15.56 7.00 11.45 
3 2 3 7 

61.58 66.17 62.7?. 60.33 
9.34 10.65 13.03 9,97 
12 12 14 6 



M 42.33 46.75 39.00 55.67 

o- 14.64*14.20 16.67 

H 6 4 16 



Intar 1 
form E 



Total M 



23-25 M 
or 
n 



- 66.00 71,00 72.67 

- 14.14 9.90 29.69 
0 2 2 3 



18-22 M 44.20 58.67 51.10 61.33 

9» 19.52 30.01 22.20 22.32 

3 5 3 10 9 

5-17 M 48.00 37.75 53.33 55.17 

<r* 19.92 14.66 16.65 25.13 

* 3 4 3 6 

Total M 
o* 
H 



Total 

56*31 
9.17 
13 

47.66 
10.76 
67 

40.03 
11.38 
29 



57.50 
15.28 
6 

58.30 
17.13 
10 

47.80 
13.77 
20 



69.27 
10.42 
15 

63.02 
10.93 

a 

47.88 

15.20 
17 



70.29 
18.78 
7 

54.07 
22.24 
27 

49.13 
19.91 
16 




WiamUHL 
4-12 13-17 18-22 23-25 

61.00 57.25 45.75 
8.89 9.57 7.68 
3 4 4 

52.86 50.71 44.00 
9.99 7.00 — 
7 17 1 

33.38 32.75 31.50 
11.04, 7.09 3.54 
3,3 4 2 



64.63 72.60 69.00 47.00 

18.56 21.40 14.63 — 

27 5 4 1 

57.10 56.88 59.90 64.20 

17.39 10.93 12.33 9.09 

29 17 21 5 

41.89 42.22 43.00 54.00 

18.45 11.60 12.73 3.tt 

9 9 2 3 



□ 



68.75 
13.25 
12 



83.33 63.40 63.40 
11.02 12.88 22.20 
3 5 5 



58.86 37.67 47.00 66.44 
18.40 24.79 17.20 19.84 
14 3 5 9 

49.00 29.00 42.60 46.00 

4.24 23.35 14.83 

12 5 5 



64.83 76.00 67.40 64.20 

15.87 20.07 4.67 12.21 

12 3 5 5 

56.74 59.60 61.50 57.30 

21.29 7.57 17.92 4.95 

19 5 6 2 

34.67 49.00 34.25 

19.50 — 13.77 

3 10 4 



Total 


TOTAL 


55.89 


56.00 


10.99 


10; 47 


38 


51 


50.28 


48.68 


10.26 


10.60 


43 


110 


35.31 


37.67 


9.13 


10.50 


29 


58^ 




47.47 




12.39 




219 


65.70 


64.56 


18.33 


18.00 


37 


43 


58.36 


58.35 


14.07 


14.36 


72 


82 


43.70 


45.60 


14.07 


13.92 


23 


43 




56.68 




16.71 




168 


68.36 


68.70 


15.45 


13.64 


25 


40 


57.10 


60.57 


20.36 


15.69 


31 


75 


42.31 


45.47 


17.19 


16.05 


13 


30 




59.69 




17.14 




145 


66.56 


67.38 


13.84 


' 14.79 


25 


32 


58.13 


56.27 


18.08 


20.01 


32 


59 


36.25 


44.83 


14.71 


19.05 


8 


^4 




56.97 




19.95 




115 
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TAILS C.7 Aaalyeie of cora^iamea** of 3A? Vocabulary , Reading Comprehension , acl Math Total 
(corresponding to data of Table* C.6e, C.6b, S C.6c)* 



low 

* ; # 



2a 

2b 



6a 

6b 



7 
8 

•9 
10 



11 
12 



SAT VOCABOLART 



Sources of 
Variances 


COHORT A 


COHORT B 


Taar 1 
Grada 1 


Taar 2 
Grada 2 


Taar 1 
Grada 3 


Taar 2 
Grada 4 




1 


df 


P 


r 


df 


P 


? 


df 


P 


t 


df 


P 


CoYarietee 
Smi Total 


























8.63 


1 


•00 


•38 


1 


• 54 


11.40 


1 


.00 


4.21 


1 


.0* 


Hrs/wfc inetruction 


























o English 


16,47 


1 


.00 


.10 


1 


• 76 


•02 


1 


•90 


2.45 


1 


.li 


o Math 


























Yr. 1 SAT score 




- 




139.44 


1 


.00 


_ 






87.50 


1 


.oc 


Main Effects 


























A. Englieh SOPR 


47.0* 


2 


.00 


1.08 


2 


•34 


7.13 


2 


•00 


2.29 


2 


.13 


B. Indian SO PR 


1.13 


3 


.34 


.72 


3 


.54 


1.51 


3 


.22 


•59 


3 


.6: 


C. t Englieh used in 


























instruction la: 
























.3? 


o Kou-T4mguege arts 


4*53 


1 


• 03 


.51 


1 


• 48 


.69 


1 


.41 


.76 


1 


o Hath 


























Interactions 
























.81 


AB 


.33 


6 


.92 


2.37 


6 


• 03 


•19 


6 


.97 


.50 


6 


AC 


•31 


2 


.27 


.19 


2 


• 82 


2.92 


2 


.06 


5.29 


2 


.01 


BC 


.21 


3 


.89 


.42 


3 


.74 


2.06 


3 


.11 


1.85 


3 


.14 


ABC 


•52 


6 


.17 


•48 


5 


.79 


1.87 


6 


.09 


1.67 


6 


•1^ 


Residual variance 




207 






157 






106 






97 




TOTAL 




232 






182 






137. 






123 





* Nnabers of cases 

K 

168-233 
115-150 




Saa Tables C.6e, 
C.6b, and C6.c 
for dataila 



** Each of the 12 analyses of covariance is baaed on lletvlse data. 
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UhU C.7 (ConO 



SAT tlAOIWC COKMHBBIOW 



COMMIT A. 



Tft«r 1 
-gtftdft 1 



Tftftr 2 
Grada 2 



df p t df p 

10.05 1 .00 2.94 i .09 

3.44 1 .06 . 21 1 .63 

240.33 1 .00 



33.02 2 .OOi 1.12 2 .33 
i;79 3 .15 1.62 3 .19 



1.91 1 .17 



•93 6 
.06 2 
.77 3 

Tr$4~-6' 



207 
232 



.78 

.9f 
.51 
~09 



2.02 1 .16 



1.78 
.89 
.19 
.90 



.11 
.41 
.90 
.48 



1ST 

182 



Tar 1 
Grad* 3 



r df p 

28.82 1 .00 
.33 1 .47 



23.02 2 .00 
•06 3 .98 



.79 1 .38 



.66 6 

•CO 2 

1.03 3 

1.61 6 



124 
149 



Tar! 
Ctftdft 4 



? df p 

4.23 1 .04 

8.62 1 .00 

127.02 X .00 



1.14 2 .32 
. .12 3 .93 



.80 1 .37 
.17 



.68 .80 

.55 .23 

.38 1.77 

.15 .66 



6 .57 

2 .80 

3 .16 
6 .66 



115 
141 



SAT MATH T0t4L 



corotr k 



T*ar 1 
Sexto 1 



? df p 
44.22 1 .00 
.08 1 .78 



F df p 

2.44 1 .12 

.05 1 .82 

123.62 1 .00 



33.58 2 .00 
.17 3 .91 



•68 



.30 
.69 



6 .94 
2 .50 



.90 3 .44 
.92 6 .48 

193 
218 



TMt 2 
Grad* 2 



1.69 2 
.26 3 



.89 
.36 
.58 
.80 



.19 
• 86 



.39 1 .53 



.50 
.10 
.63 
.55 



142 
167 



COHORT B 



Tur 1 
Gcada 3 



? df p 
66.29 1 .00 
•19 1 .67 



12.89 2 .00 
.20 3 .90 



1.28 1 .25 



.96 
1.27 
2.04 
1.24 



.46 
.28 
.11 
.29 



119 
144 



Tur 2 
GrmiU 4 



ton , 
# 



? df p 

5.34 1 .02 

18.17 1 .00 

79.16 1 .00 



.68 2 .51 
•26 3 -.86 



.58 1 .45 



.60 
.43 
.64 
1.37 



6 .72 

2 .65 

3 '.39 

4 .25 



90 
114 



2« 

2b s 



6ft c 

6b 



11 

12 



4X120/2.86 



ERIC 



100 



As for the main effects, ail of the year 1 values of P for English SOPR are 
significant at the "sOtT level. It thus appears that English SOPR plays 
somewhat the same role in Year 1 as the SAT variables play in the Year 2 
analyses. None of F values for Indian SOPR are significant at the .05 level. 
In regard fo the percentage-of-time-ih-English variable, only one of 12 F 
values (Vocabulary, grade 1) is significant* The percentage-of-tiiae variable 
for the Vocabulary criterion, it will be recalled, is percentage of English in 
native science, .tnd social studies instruction. That it has a significant F 
ratio just for grade 1 is probably due to the fact that it is in grade 1 that 
the students 9 English is poorest, and that therefore the benefit they receive 
from hearing English used routinely is greatest. This interpretation is 
supported by the direction of the relationship between vocabulary and the 
percentage-in-English variable; the correlation (for the Vocabulary Grar"^ 1 
analysis) is a substantial negative one (-.283). 

. Rather surprisingly, the language used in math instruction appears to be 
somewhat irrelevant with respect to math achievement; none of the four F 
ratios even comes close to significance. Perhaps the advantages and 
disadvantages of using the Indian language cancel each other out. 

As for the interactions, there are only sporadic instances of statistical 
significance (2 out of 48, both of them occurring for the Vocabulary 
criterion); we attach no particular importance to them. 

C.2d FOCUSED SUBGROUP ANALYSIS 

The three preceding sections of this appendix (i.e., the sections on 
regression analysis, path analysis, and covariance analysis) dealt with 
statistical analyses based on all available cases. Now we turn to a set of 
analyses in which each analysis is based on a tightly defined 
mutlidimensionally homogeneous group of cases, which is split into two 
subgroups differing in one important dimension: the percentage of instruction 
that is presented in English. These analyses really amount in effect to a 
modification and simplification of the covariance analysis approach described 
above. We compare it to covariance analysis because it considers the results 
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for homogeneous subgroups* But instead of making the groups homogeneous with 
respect to a particular variable by statistical control (i.e., using the 
variable as a covariate) it was accomplished directly, by classifying the 
students on that variable. More specifically, this fourth approach, as 
applied in the present instance, consisted in obtaining criterion me&ns for 
subgroups homogeneous with respect to four dichotomized variables: English 
SOFR, Indian SOFR, Raven, and percentage-use-of-English-in-instruction, this 
last-named variable again being for instruction in math, science, and social 
studies in the case of the Vocabulary and Reading Comprehension criterion 
variables, and for instruction in math in the case of the Math criterion. 
Thus this approach is about the same as that resulting in Tables C.6a, C.6b, 
and C.6c, except that in those tables the SOPRs had more than two categories 
each, and Raven was not a classification variable. The final step in these 
focused subgroup analyses was to compare criterion variable means on subgroups 
paired in such a way that the pair is homogeneous on the SOPRS and the Raven, 
but one subgroup of the pair is high and the other low, on the dichotomized 
percentage~of-use-of-English variable. Table C.8a, shows the Vocabulary and 
Reading Comprehension results for the 4-way breakdown of cases; Table C.8b 
shows the corresponding data for Math. It will be noted that the subgroups 
are organized into the pairs to be compared. 

These two tables permit exploration of the following three hypotheses. 

1. For the Vocabulary criterion 

Students will learn more English vocabulary if English is used when 
they are being taught math, science, and social studies than if some 
other language is used. 

2. For the Reading Comprehension criterion 

Students will improve their English reading skills more if English is 
used in the course of theif math, science, and social studies 
instruction than if some other language is used. 
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TAfttK- C.C* aad ,«aadaM <tartatloaa of Taar 1 3 A? Vocabulary and Raadlnt, Coaarahantlon 

for ttudanta claaaifiad on tha baala of SOPH icorti, Uvin, tod oarcantaga uaa of 
tht- tatliau Un*ua«a la taachint aath, •cianct. and aoolal atudlaa 

Koto j All of eh* claaatficatioa variablaa hava baaa dichotoaiitd Into a "plua" catatory and * 
- adnua" catatorr* 




X UM 

Cagllah 



SAT 
Vocab. 



SAT 
Ida. Cgg£g 



Hlr 
# 









■77 






x 7 


16 


25.9 


8.7 


16 


IV 












17.4 


2.9 


11 


24.6 


7.1 


U 




+ 


+ 




18 




19.6 


5.4 


11 


27.8 


6.3 


11 


2V 












23 1 


6.1 


7 


29.7 


10.5 


7 




+ 






57 




24.2 


5.4 


37 


27,4 


8.6 


37 


3V 












20 0 




20 


24.0 


9.0 


20 






- 


- 


49 




21.0 


5.4 


20 


24.6 


6.9 


20 


4V 












20 2 


4.7 


29 


23.6 


8.8 


29 




- 






16 




14.8 


1.7 


4 


18.8 


6.9 


4 


5V 












l«t x 


3.6 


12 


22.0 


7.4 


12 




- 




- 


18 




14.7 


3.3 


7 


20.6 


4.6 


7 


5V 












14.5 


2.6 


11 


16.2 


3.4 


u 






- 


+ 


15 




15.6 


3.3 


7 


20.3 


6.5 


7 


7V 












14.2 


4.3 


8 


17.9 


7.4 


8 




- 


- 


- 


33 




16.1 


3.1 


19 


18.3 


5.4 


19 


8V 












15.1 


4.3 




15.8 


4.7 


14 














14.5 


5.9 


10 


33.2 


6.9 


10 


lR 












16.2 


3.7 


12 


35. 8 


U.7 


17 




+ 






15 


+ 


12.5 


3.7 


4 


29.0 


13.0 


4 


2JL 












15.3 


4.0 


11 


30.4 


8.5 


15 




+ 






30 




20.1 


6.0 


15 


38.4 


12.0 


16 


3R 












17.4 


5.2 


15 


31.0 


9.2 


16 




+ 






32 




16.1 


6.5 


13 


26.3 


10.2 


13 


AH 












15.3 


4.7 


19 


29.0 


9.9 


20 






+ 


+ 


4 


+ 


15.5 


7.8 


2 


30.0 


14.1 


2 


5K 












11.5 


4.9 


2 


22.2 


8.5 


6 






+ 




10 


+ 


9.7 


3.1 


6 


20.7 


6.7 


6 


6* 












14.0 


3.7 


4 


18.2 


4.6 


6 










3 








0 






0 


7R 












12.3 


4.0 


3 


23.3 


12.9 


3 










16 




11.8 


5.2 


6 


22.2 


9.5 


6 














U.8 


2.3 


10 


22.6 


5.7 


16 





*Tht claaalflcatien variablaa art dichotoaitad aa follow* t 





"Minua" 


"Plua" 




catatory 


catatory 


togllsh SOP* 


5-18 


19-25 


Indian S0PR 


4-18 


19-25 


Rartn 






CM (Cohort A) 


0-20 


21-36 


SW (Cohort B) 


0-28 


29-60 


X oh of tntlU4 




Mora than 89. OX 


la Cohort A 


Up to 89. OX 


Xa Cohort B 


Up to 88 *0X 


Mora than 88. OX 



r ?«rcantata uao of Sngllah in taachln* math, sctance, tad 3 octal atudtaa. 
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TABLE G.8b Hum: tad ittndard dartatlooa of Yaar 1 SAX Math Total (or studaata elaiilflad on 
tha baala of SOP* acoraa, Una, ;aad parcaaeaft ust of cha Eafllah laoguaga la 
teaching aath 

..oft All of cha clarification vartablaa barn baan dlchototltad into a "plua" casagory and a 
"ninna" catagory* 



T una of 



Cohort A - Crarta 1 



SAT Math Total 

Cohort B - Crada 3 



Pair 
# 





J J 




N 


-0- 


it 


M 




If 








a 


51.5 


1Q.B 


19 


"Oa.a 


13. 5 


13 


JL 








54.3 


7.3 


7 


69.4 


13.4 


11 










49.3 


11.3 


17 


44.0 


9.7 


5 


2 








49.3 


U.3 


17 


60.1 


9.1 


14 










56.3 


10.1 


49 


72.1 


14.6 


19 


3 








50.3 


8.0 


7 


68.5 


U.9 


12 










51.2 


U.O 


30 


51.8 


13.3 


12 


4 








42;4' 


9.2 


18 


62.6 


9.8 


19 










41.2 


U.6 


5 


49.7 


31.6 


3 


5 








50.2 


8.4 


6 


59.4 


18.5 


5 






¥ - 




37.6 


9.7 


9 


41.2 


13.3 


8 


6 








35.6 


12.5 


5 


49.6 


6.5 


4 










41.0 


9.7 


10 






0 


7 








47.0 


7.4 


5 


60.7 


12,0 


3 










35.6 


9.4 


25 


44.9 


24.7 


7' 


8 








32.4 


9.9 


7 


41.5 


14.1 


8 





*Tha cUaaification varlablaa ara dichotoaitad aa follovat 



Enaliah SOW 
Iadian S0PR 
Kavan 

CFN (Cohort A) 
SPM (Cohort B) 



•Mltma* 
catagory 

5-18 
4-16 

0-20 
0-28 



•Plua* 
catagory 

19-25 
19-25 

21-36 
29-60 



Z uaa of English 0p to 89. OX Mora than 89. OX 
*Ptrcant*gt uaa of Engiiah la taaching aath. 
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3. For the Math criterion 

Students will learn math better if they are taught it in a language 
they understand well than if the instruction is presented in a 
language in which, their proficiency is limited* (In connection with 
this hypothesis it should be recognized that to the extent that the 
hypothesis is supported for math* it can reasonably be inferred by 
extension \that analogous hypotheses regarding the language of 
instruction for teaching science and social studies would also be 
supported.) 

To check on whether this third hypothesis is supported by the empirical 
data, it Is necessary to know in which language - English or an Indian 
language - the students are more proficient in their speech and comprehension 
of the. spoken language. Since the same five scales (Comprehension, Fluency, 
Vocabulary, Pronunciation, and Grammar) were used on the two SOFRs, and since 
the wording of each of the five 5-point scales was identical for the two 
SOPRs, dichotomizing the two SOFRs at exactly the same point makes it 
reasonable to assume that if a student is in the high category on one and the 
low category on the* other, he (she) has a better command of the language for 
which his SOPR is in the high category. Therefore in the case of the Math 
criterion, pairs 3,4,5, and 6 oh Table C.8b - i.e., the four pairs for which 
one of the two SOPRs is high and the other low - are critical in determining 
whether there, is any evidence in support of hypotheses 3, stated above. For 
the other two criterion variables (Vocabulary and Reading Comprehension) all 
eight kinds of pairs shown in Table C.8a are relevant. Therefore for the 
Vocabulary and Reading Comprehension criteria the Cohort A and Cohort B 
results are summarized in Table C.9 for all eight kinds of pairs. For the 
math criterion, Table C.9 includes only the data for the four critical pairs 
(#3, #4, #5, and #6). 

For both the Vocabulary criterion and the Reading Comprehension criterion 
there are 15 critical comparisons - (15 pairs with data for each category of 
the dichotomy on percentage use of English). For the Math criterion there are 
only .8 critical comparisons (four kinds of pairs, two cohorts for each). As 
shown in the bank, of three columns at the right of Table C.9, 9.5 of the 15 
comparisons (a tie was counted as .5) for the Vocabulary criterion, 9 of the 
15 for Reading Comprehension, and 6 of the 8 for Math turn out to support the 

105 



DEVELOPMENT ASSOCIATES, .INC. 



C-28 



TAILS C;9 Chocking tha labia C8s tad C.8b <Utt t|tluc thraa hypotbssss 

Hypothesis 1* Toesbularr acoro> srs higher if lmtructloa la uth, scieact and 
social studies. is mostly la English than if another language Is 
used extensively. 

Hypothesis 2* leading Comprehension acoraa ara highar if Instruction In aath, 
sclscce, and social stadias is mostly la English than if another 
Uaftaaga is used extensively. 

Hypotheaie 3. Math acoraa ara highar if aath instruction is mostly la r* languaga 

tha student understands wall than if anothar language la uaad extensively. 
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Ho. of pairs for which comparison data ara svailabla 
Ho. of pairs which fit tha* hypothasis 



13 13 3 
9.5** 9 6 



•Identified as in Tables G.Se and C. 8b. 
**A tin is counted as .5. 
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corresponding hypocheses: hypothesis 1 for Vocabulary, hypotheses 2 for 
Reading Comprehension, and hypothesis 3 for Math. Thus the Hypotheses are 
supported, though by no means proven, by the, Table C. 9 data. The results, 
though in the right direction for supporting the hypotheses, are not 
significantly different from chance. It is important to recognize that 
alternative hypotheses could almost certainly be formulated with which the 
empirical data would be at least as compatible^ 



3. SUMMARY OP FINDINGS AMD IMPLICATIONS 



Two of tlie main findings of the analyses described in this appendix are (1) 
that the best predictors of achievement are some of the initial status 
variables - the Raven, the English SOPR, and previous scores on the SAT - and 
(2) that the instructional variables investigated (amount of instruction in 
specific subjects, and language of instruction) are not significantly related 
to outcomes (or at least that significant relationships were not manifested in 
the sample studied). 

The English SOPR turns out to be an even better indicator of how well the 
children in this study will do in school (as measured by three SAT subtests - 
Vocabulary, Reading Comprehension, and Mathematics) than the measure of 
academic aptitude used - the Raven Progressive Matrices test. This is 
somewhat contrary to the usual situation in which academic performance is 
being predicted. Not surprisingly, in educational research the best predictor 
of academic performance is typically found to be academic aptitude . The 
somewhat different findings of the present study are in line with expectation, 
however; they are undoubtedly a consequence of the fact that so many of 
students in the study fall in the limited-English-proficient category. These 
students thu*. ard % iq+ able to perform at the level that would otherwise be 
expected on the basis of their academic potential (as indicated by the 
itf" ^5; aptitude test). It seems reasonable to suppose that as the children 
<6 through school, and as their English improves, the Raven will 

i>ecom<i a better predictor and the importance of the English SOPR will 
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decline* Indeed that process is already under way as revealed by a comparison 
of \the Cohort A and Cohort B correlations of the Raven with criterion data 
(these r f s are considerably higher for Cohort B, which is two years further 
along in school) and a similar comparison of cohorts on the English SOPR 
correlations (which are somewhat lower for Cohort B than A). 

The Indian SOPR doe3 not appear to be closely related to achievement but 
what relationships there are, both in terms of regression weights and 
correlation coefficients, tend to be largely negative. 

In line with the well-known aphorism that past performance is the best 
predictor of future performance we have found that the year 1 score on an SAT 
test is an excellent predictor of the year 2 SAT score on the same subject. 
In the year 2 multiple regression analyses, in which it was feasible to 
incorporate year 1 SAT scores as independent variables, they supplanted 
English SOPR and the best predictors. 

However the past-status and past-performance variables we have been talking 
about in this section - i.e., Raven, SOPRs, previous SAT's - do not account 
for all the variance. As a matter of fact, as can be seen from Table C.2 by 
adding up the values in the Z\ columns (i.e., the percentages of variance 
accounted for) for all rows except instructional variables, only about 45 to 
65 percent of the year 2 SAT variance is accounted for. If a modest 
percentage is added for the effects of home and family variables that are 
independent of the variables already taken into account, that leaves a 
substantial percentage unaccounted for. In other words there is plenty of 
room for instructional variables to have an effect. The effect they are 
having, however, apnears to be surprisingly slight. Our investigation of the 
effects of number of hours of instruction, and loice of language in which to 
present instruction, showed some differences, but generally they were not 
large enough to be statistically significant. 
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What should be concluded -from^ the failure to demonstrate statistical 
significance for findings concerning instructional methodology? One view, 
probably an unduly pessimistic one, is that nothing the schools do matters - 
that whether the child is successful in school is almost wholly determined by 
conditions beyond the school's or teacher's control* 

A second, explanation and probably an important one, of the failure to show 
statistically significant effects for instructional methodology variables is 
the fact that so little of the instruction is p ~sented in the Indian 
language* (The grade 1 median is only about 10 percent.) 

As for the independent variables in such categories as "school variables** 
and "teachers variables'* (categories from which the instructional methodology 
variables discussed above have:, been excluded),, their, correlations with. the. 
criterion measures (achievement test scores) tend to be either low or 
unstable* 

Among the home-and-family variables the ones that have the highest 
correlations with the criterion variables are: (1) parent's use of English in 
the home, (2) the amount of time the child devotes to reading (apart from 
homework assignments) and (3) parental expectations regarding how far the 
child will go in school* 
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